megfile.http module

megfile.http.get_http_session(timeout: int | Tuple[int, int] | None = (60, 86400), status_forcelist: Iterable[int] = (500, 502, 503, 504)) Session[source]
megfile.http.http_exists(path: str | BasePath | PathLike, followlinks: bool = False) bool[source]

Test if http path exists

Parameters:
  • path – Given path

  • followlinks (bool, optional) – ignore this parameter, just for compatibility

Returns:

return True if exists

Return type:

bool

megfile.http.http_getmtime(path: str | BasePath | PathLike, follow_symlinks: bool = False) float[source]

Get Last-Modified time of the http request on the given http_url path.

If http response header don’t support Last-Modified, will return None

Parameters:
  • path – Given path

  • follow_symlinks – Ignore this parameter, just for compatibility

Returns:

Last-Modified time (in Unix timestamp format)

Raises:

HttpPermissionError, HttpFileNotFoundError

megfile.http.http_getsize(path: str | BasePath | PathLike, follow_symlinks: bool = False) int[source]

Get file size on the given http_url path.

If http response header don’t support Content-Length, will return None

Parameters:
  • path – Given path

  • follow_symlinks – Ignore this parameter, just for compatibility

Returns:

File size (in bytes)

Raises:

HttpPermissionError, HttpFileNotFoundError

megfile.http.http_open(path: str | BasePath | PathLike, mode: str = 'rb', *, encoding: str | None = None, errors: str | None = None, max_concurrency: int | None = None, max_buffer_size: int = 134217728, forward_ratio: float | None = None, block_size: int = 8388608, **kwargs) BufferedReader | HttpPrefetchReader[source]

Open a BytesIO to read binary data of given http(s) url

Note

Essentially, it reads data of http(s) url to memory by requests, and then return BytesIO to user.

Parameters:
  • path – Given path

  • mode – Only supports ‘rb’ mode now

  • encoding – encoding is the name of the encoding used to decode or encode the file. This should only be used in text mode.

  • errors – errors is an optional string that specifies how encoding and decoding errors are to be handled—this cannot be used in binary mode.

  • max_concurrency – Max download thread number, None by default

  • max_buffer_size – Max cached buffer size in memory, 128MB by default

  • block_size – Size of single block, 8MB by default. Each block will be uploaded or downloaded by single thread.

Returns:

BytesIO initialized with http(s) data

megfile.http.http_stat(path: str | BasePath | PathLike, follow_symlinks=True) StatResult[source]

Get StatResult of http_url response, including size and mtime, referring to http_getsize and http_getmtime

Parameters:
  • path – Given path

  • follow_symlinks – Ignore this parameter, just for compatibility

Returns:

StatResult

Raises:

HttpPermissionError, HttpFileNotFoundError

megfile.http.is_http(path: str | BasePath | PathLike) bool[source]

http scheme definition: http(s)://domain/path

Parameters:

path – Path to be tested

Returns:

True if path is http url, else False