lib.sedna.common.file_ops

FileOps class.

Module Contents

Classes

FileOps

This is a class with some class methods
class lib.sedna.common.file_ops.FileOps[source]

This is a class with some class methods to handle some files or folder.

classmethod make_dir(cls, *args)[source]

Make new a local directory.

Parameters

args (*) – list of str path to joined as a new directory to make.

classmethod get_file_hash(cls, filepath)[source]
classmethod clean_folder(cls, target, clean=True)[source]

clean the target directories. create path if target not exists, initial path if clean be True

Parameters

  • target (list) – list of str path need to clean.

  • clean (bool) – clear target if exists.

classmethod delete(cls, path)[source]
classmethod make_base_dir(cls, *args)[source]

Make new a base directory.

Parameters

args (*) – list of str path to joined as a

new base directory to make.

classmethod join_path(cls, *args)[source]

Join list of path and return.

Parameters

args (*) – list of str path to be joined.

Returns

joined path str.

Return type

str

classmethod remove_path_prefix(cls, org_str: str, prefix: str)[source]

remove the prefix, for converting path in container to path in host.

classmethod dump_pickle(cls, obj, filename)[source]

Dump a object to a file using pickle.

Parameters

  • obj (object) – target object.

  • filename (str) – target pickle file path.

classmethod load_pickle(cls, filename)[source]

Load a pickle file and return the object.

Parameters

filename (str) – target pickle file path.

Returns

return the loaded original object.

Return type

object or None.

classmethod copy_folder(cls, src, dst)[source]

Copy a folder from source to destination.

Parameters

  • src (str) – source path.

  • dst (str) – destination path.

classmethod copy_file(cls, src, dst)[source]

Copy a file from source to destination.

Parameters

  • src (str) – source path.

  • dst (str) – destination path.

classmethod dump(cls, obj, dst=None) → str[source]
classmethod load(cls, src: str)[source]
classmethod is_remote(cls, src)[source]
classmethod download(cls, src, dst=None, unzip=False) → str[source]
classmethod upload(cls, src, dst, tar=False, clean=True) → str[source]
classmethod is_local(cls, src)[source]
classmethod gcs_download(cls, src, dst)[source]

todo: not support now

classmethod gcs_upload(cls, src, dst)[source]

todo: not support now

classmethod s3_download(cls, src, dst)[source]
classmethod s3_upload(cls, src, dst)[source]
classmethod http_download(cls, src, dst)[source]

Download data from http or https web site.

Parameters

  • src (str) – the data path

  • dst (str) – the data path

Raises

FileNotFoundError – if the file path is not exist, an error will raise

classmethod exists(cls, folder)[source]

Is folder existed or not.

Parameters

folder (str) – folder

Returns

folder existed or not.

Return type

bool

classmethod obj_to_pickle_string(cls, x)[source]
classmethod pickle_string_to_obj(cls, s)[source]