Advanced User Guide

Define Custom Protocol

Megfile support custom protocols. You can define your own protocol class like this:

# custom.py
import io
from typing import IO, AnyStr

from megfile.interfaces import URIPath
from megfile.smart_path import SmartPath

@SmartPath.register
class CustomPath(URIPath):

    protocol = "custom"

    def open(self, mode: str = 'rb', **kwargs) -> IO[AnyStr]:
        return io.BytesIO(b'test')

    ...

• protocol = "custom" is the name of your custom protocol. Then your path will be like custom://path/to/file.

• Implement methods

  • URIPath provide some properties and methods like path_with_protocol, path_without_protocol, parts, parents and you can use them. You can read more about them in megfile.pathlike.URIPath.

  • smart methods will call your CustomPath‘s methods automatically, if you have implemented the corresponding method. For example: if you implement CustomPath.open, smart_open will call it when path is custom://path/to/file. You can find the corresponding class methods required for smart methods in megfile.smart_path.SmartPath.

• You must import your custom python file before you use smart methods. You must make the decorator @SmartPath.register effective. Like this:

from custom import CustomPath
from megfile import smart_open

with smart_open("custom://path/to/file", "rb") as f:
    assert f.read() == b'test'

Glob Pattern

The glob module finds all the pathnames matching a specified pattern according to the rules.

Patterns are Unix shell style:

pattern	meaning
*	matches any characters but ‘/’
**	matches everything
?	matches any single character
[seq]	matches any character in seq
[!seq]	matches any char not in seq
{seq1,seq2}	matches seq1 or seq2