megfile.http module
- megfile.http.get_http_session(timeout: int | Tuple[int, int] | None = (60, 86400), status_forcelist: Iterable[int] = (500, 502, 503, 504), params: Dict[str, str] | None = None, headers: Dict[str, str] | None = None, cookies: Dict[str, str] | None = None, trust_env: bool = True, **kwargs) Session[source]
- megfile.http.http_exists(path: str | BasePath | PathLike, followlinks: bool = False) bool[source]
Test if http path exists
- Parameters:
path – Given path
followlinks (bool, optional) – ignore this parameter, just for compatibility
- Returns:
return True if exists
- Return type:
bool
- megfile.http.http_getmtime(path: str | BasePath | PathLike, follow_symlinks: bool = False) float[source]
Get Last-Modified time of the http request on the given http_url path.
If http response header don’t support Last-Modified, will return None
- Parameters:
path – Given path
follow_symlinks – Ignore this parameter, just for compatibility
- Returns:
Last-Modified time (in Unix timestamp format)
- Raises:
HttpPermissionError, HttpFileNotFoundError
- megfile.http.http_getsize(path: str | BasePath | PathLike, follow_symlinks: bool = False) int[source]
Get file size on the given http_url path.
If http response header don’t support Content-Length, will return None
- Parameters:
path – Given path
follow_symlinks – Ignore this parameter, just for compatibility
- Returns:
File size (in bytes)
- Raises:
HttpPermissionError, HttpFileNotFoundError
- megfile.http.http_open(path: str | BasePath | PathLike, mode: str = 'rb', *, encoding: str | None = None, errors: str | None = None, max_workers: int | None = None, max_buffer_size: int = 134217728, block_forward: int | None = None, block_size: int = 8388608, **kwargs) BufferedReader | HttpPrefetchReader[source]
Open a BytesIO to read binary data of given http(s) url
Note
Essentially, it reads data of http(s) url to memory by requests, and then return BytesIO to user.
- Parameters:
path – Given path
mode – Only supports ‘r’ or ‘rb’ mode now
encoding – encoding is the name of the encoding used to decode or encode the file. This should only be used in text mode.
errors – errors is an optional string that specifies how encoding and decoding errors are to be handled—this cannot be used in binary mode.
max_workers – Max download thread number, None by default, will use global thread pool with 8 threads.
max_buffer_size – Max cached buffer size in memory, 128MB by default. Set to 0 will disable cache.
block_forward – How many blocks of data cached from offset position
block_size – Size of single block, 8MB by default. Each block will be uploaded or downloaded by single thread.
- Returns:
A file-like object with http(s) data
- megfile.http.http_stat(path: str | BasePath | PathLike, follow_symlinks=True) StatResult[source]
Get StatResult of http_url response, including size and mtime, referring to http_getsize and http_getmtime
- Parameters:
path – Given path
follow_symlinks – Ignore this parameter, just for compatibility
- Returns:
StatResult
- Raises:
HttpPermissionError, HttpFileNotFoundError