lib.sedna.common.file_ops

FileOps class.

Module Contents

Classes

FileOps

This is a class with some class methods

class lib.sedna.common.file_ops.FileOps

This is a class with some class methods to handle some files or folder.

classmethod make_dir(*args)

Make new a local directory.

Parameters:

args (*) – list of str path to joined as a new directory to make.

classmethod get_file_hash(filepath)

classmethod clean_folder(target, clean=True)

clean the target directories. create path if target not exists, initial path if clean be True

Parameters:

• target (list) – list of str path need to clean.

• clean (bool) – clear target if exists.

classmethod delete(path)

classmethod make_base_dir(*args)

Make new a base directory.

Parameters:

args (*) – list of str path to joined as a

new base directory to make.

classmethod join_path(*args)

Join list of path and return.

Parameters:

args (*) – list of str path to be joined.

Returns:

joined path str.

Return type:

str

classmethod remove_path_prefix(org_str: str, prefix: str)

remove the prefix, for converting path in container to path in host.

classmethod dump_pickle(obj, filename)

Dump a object to a file using pickle.

Parameters:

• obj (object) – target object.

• filename (str) – target pickle file path.

classmethod load_pickle(filename)

Load a pickle file and return the object.

Parameters:

filename (str) – target pickle file path.

Returns:

return the loaded original object.

Return type:

object or None.

classmethod copy_folder(src, dst)

Copy a folder from source to destination.

Parameters:

• src (str) – source path.

• dst (str) – destination path.

classmethod copy_file(src, dst)

Copy a file from source to destination.

Parameters:

• src (str) – source path.

• dst (str) – destination path.

classmethod dump(obj, dst=None) → str

classmethod load(src: str)

classmethod is_remote(src)

classmethod download(src, dst=None, unzip=False) → str

classmethod upload(src, dst, tar=False, clean=True) → str

classmethod is_local(src)

classmethod gcs_download(src, dst)

todo: not support now

classmethod gcs_upload(src, dst)

todo: not support now

classmethod s3_download(src, dst)

classmethod s3_upload(src, dst)

classmethod http_download(src, dst)

Download data from http or https web site.

Parameters:

• src (str) – the data path

• dst (str) – the data path

Raises:

FileNotFoundError – if the file path is not exist, an error will raise

classmethod exists(folder)

Is folder existed or not.

Parameters:

folder (str) – folder

Returns:

folder existed or not.

Return type:

bool

classmethod obj_to_pickle_string(x)

classmethod pickle_string_to_obj(s)