CLI

aiomegfile ships with a CLI named amf.

Installation

pip install "aiomegfile[cli]"

Quick Examples

amf ls ./data
amf ls s3://my-bucket/prefix -l
amf cp -r ./data s3://my-bucket/backup
amf sync ./data s3://my-bucket/backup --progress-bar
amf cat https://example.com/data.txt
printf 'payload' | amf to s3://my-bucket/stdin-demo.txt

Command Groups

The CLI covers several common workflows:

Inspection: ls, ll, stat, size, mtime, md5sum
Transfer: cp, mv, sync, rm, mkdir, touch
Streaming: cat, head, tail, to, edit
Configuration: config
Shell integration: completion

Protocol Support

The CLI works with the same backend registry as the Python API:

local paths and file://
s3://
http:// and https://
sftp://
stdio://
hdfs:// when the HDFS extra is installed
webdav:// and webdavs:// when the WebDAV extra is installed

Configuration

Helper commands are available for common configuration files:

amf config s3 <access_key> <secret_key> --profile-name default
amf config hdfs http://namenode:9870 --profile-name prod --user hdfs
amf config alias datasets s3://company-datasets/
amf config env MEGFILE_MAX_WORKERS=16

Completion

The CLI can append completion setup to your shell config:

amf completion bash
amf completion zsh
amf completion fish

Notes

sync modifies the destination tree to match the source.
cp supports recursive copies with -r and progress bars with -g.
to reads from stdin and writes to the target path.
cat, head, and tail are useful for quick remote inspection.

Reference

amf

Client for aiomegfile.

Usage

amf [OPTIONS] COMMAND [ARGS]...

Options

--debug: Enable debug mode.

--log-level <log_level>

Set logging level.

Options:: DEBUG | INFO | WARNING | ERROR

cat

Print the file content to stdout.

Usage

amf cat [OPTIONS] PATH

Arguments

PATH: Required argument

completion

Shell completion helper.

Usage

amf completion [OPTIONS] COMMAND [ARGS]...

bash

Enable bash completion for aiomegfile.

Usage

amf completion bash [OPTIONS]

fish

Enable fish completion for aiomegfile.

Usage

amf completion fish [OPTIONS]

zsh

Enable zsh completion for aiomegfile.

Usage

amf completion zsh [OPTIONS]

config

Configuration helpers.

Usage

amf config [OPTIONS] COMMAND [ARGS]...

alias

Update alias configuration in the config file.

param path:: Config file path.
param name:: Alias name.
param protocol_or_path:: Protocol or protocol/prefix mapping.
param no_cover:: Whether to forbid overwriting existing alias.
return:: None
rtype:: None

Usage

amf config alias [OPTIONS] NAME PROTOCOL_OR_PATH

Options

-p, --path <path>: megfile config file, default is ~/.config/megfile/megfile.conf

--no-cover: Not cover the same-name config

Arguments

NAME: Required argument

PROTOCOL_OR_PATH: Required argument

env

Update env configuration in the config file.

param path:: Config file path.
param expr:: Environment assignment in the form NAME=VALUE.
param no_cover:: Whether to forbid overwriting existing env.
return:: None
rtype:: None

Usage

amf config env [OPTIONS] EXPR

Options

-p, --path <path>: megfile config file, default is ~/.config/megfile/megfile.conf

--no-cover: Not cover the same-name config

Arguments

EXPR: Required argument

hdfs

Update HDFS configuration in the config file.

param path:: Config file path.
param url:: HDFS URL.
param profile_name:: Profile name.
param user:: HDFS user.
param root:: Root path.
param token:: Auth token.
param timeout:: Request timeout.
param no_cover:: Whether to forbid overwriting existing profile.
return:: None
rtype:: None

Usage

amf config hdfs [OPTIONS] URL

Options

-p, --path <path>: hdfs config file, default is $HOME/.hdfscli.cfg

-n, --profile-name <profile_name>: hdfs config file

-u, --user <user>: user name

-r, --root <root>: hdfs path’s root dir

-t, --token <token>: token for requesting hdfs server

-o, --timeout <timeout>: request hdfs server timeout, default 10

--no-cover: Not cover the same-name config

Arguments

URL: Required argument

s3

Update S3 credentials in the AWS credentials file.

Usage

amf config s3 [OPTIONS] AWS_ACCESS_KEY_ID AWS_SECRET_ACCESS_KEY

Options

-p, --path <path>: S3 config file, default is $HOME/.aws/credentials

-n, --profile-name <profile_name>: S3 profile name

-e, --endpoint-url <endpoint_url>: endpoint-url

-st, --session-token <session_token>: session-token

-as, --addressing-style <addressing_style>: addressing-style

-sv, --signature-version <signature_version>: signature-version

--no-cover: Not cover the same-name config

Arguments

AWS_ACCESS_KEY_ID: Required argument

AWS_SECRET_ACCESS_KEY: Required argument

cp

Copy files from source to destination.

Usage

amf cp [OPTIONS] SRC_PATH DST_PATH

Options

-r, --recursive: Command is performed on all files under the specified path.

-T, --no-target-directory: Treat dst_path as a normal file.

-g, --progress-bar: Show progress bar.

--skip: Skip existed files.

Arguments

SRC_PATH: Required argument

DST_PATH: Required argument

edit

Edit a file using a local editor.

param path:: File path to edit.
param editor:: Editor command to use.

Usage

amf edit [OPTIONS] PATH

Options

-e, --editor <editor>: Editor to use.

Arguments

PATH: Required argument

head

Print the first lines of a file.

Usage

amf head [OPTIONS] PATH

Options

-n, --lines <lines>: Print the first NUM lines

Arguments

PATH: Required argument

ll

List all the objects in the path with human readable sizes.

Usage

amf ll [OPTIONS] PATH

Options

-f, --full: Displays the full path of each file.

-r, --recursive: Command is performed on all files under the specified path.

Arguments

PATH: Required argument

ls

List all the objects in the path.

Usage

amf ls [OPTIONS] PATH

Options

-l, --long: List all the objects with size, modification time and path.

-f, --full: Displays the full path of each file.

-r, --recursive: Command is performed on all files under the specified path.

-h, --human-readable: Displays file sizes in human readable format.

Arguments

PATH: Required argument

md5sum

Compute md5 checksum for a file.

Usage

amf md5sum [OPTIONS] PATH

Arguments

PATH: Required argument

mkdir

Create a directory.

Usage

amf mkdir [OPTIONS] PATH

Arguments

PATH: Required argument

mtime

Return the last modification time of a path.

Usage

amf mtime [OPTIONS] PATH

Arguments

PATH: Required argument

mv

Move files from source to destination.

Usage

amf mv [OPTIONS] SRC_PATH DST_PATH

Options

-r, --recursive: Command is performed on all files under the specified path.

-T, --no-target-directory: Treat dst_path as a normal file.

-g, --progress-bar: Show progress bar.

--skip: Skip existed files.

Arguments

SRC_PATH: Required argument

DST_PATH: Required argument

rm

Remove files from a path.

Usage

amf rm [OPTIONS] PATH

Options

-r, --recursive: Command is performed on all files under the specified path.

Arguments

PATH: Required argument

size

Return the size of a file or directory.

Usage

amf size [OPTIONS] PATH

Arguments

PATH: Required argument

stat

Return stat info for a path.

Usage

amf stat [OPTIONS] PATH

Arguments

PATH: Required argument

sync

Sync files from source to destination.

Usage

amf sync [OPTIONS] SRC_PATH DST_PATH

Options

-f, --force: Copy files forcible, ignore same files.

--skip: Skip existed files.

-w, --worker <worker>: Number of workers.

-g, --progress-bar: Show progress bar.

-v, --verbose: Show more progress log.

-q, --quiet: Not show any progress log.

Arguments

SRC_PATH: Required argument

DST_PATH: Required argument

tail

Print the last lines of a file.

Usage

amf tail [OPTIONS] PATH

Options

-n, --lines <lines>: Print the last NUM lines

-f, --follow: Output appended data

Arguments

PATH: Required argument

to

Write stdin to a file.

Usage

amf to [OPTIONS] PATH

Options

-a, --append: Append to the given file

-o, --stdout: Output to stdout as well

Arguments

PATH: Required argument

touch

Touch a file.

Usage

amf touch [OPTIONS] PATH

Arguments

PATH: Required argument

version

Return the aiomegfile version.

Usage

amf version [OPTIONS]