lmflow.args

This script defines dataclasses: ModelArguments and DatasetArguments, that contain the arguments for the model and dataset used in training.

It imports several modules, including dataclasses, field from typing, Optional from typing, require_version from transformers.utils.versions, MODEL_FOR_CAUSAL_LM_MAPPING, and TrainingArguments from transformers.

MODEL_CONFIG_CLASSES is assigned a list of the model config classes from MODEL_FOR_CAUSAL_LM_MAPPING. MODEL_TYPES is assigned a tuple of the model types extracted from the MODEL_CONFIG_CLASSES.

Attributes

MODEL_CONFIG_CLASSES
MODEL_TYPES
logger
PIPELINE_ARGUMENT_MAPPING

Classes

OptimizerNames
ModelArguments	Define a class ModelArguments using the dataclass decorator.
VisModelArguments	Define a class ModelArguments using the dataclass decorator.
DatasetArguments	Define a class DatasetArguments using the dataclass decorator.
MultiModalDatasetArguments	Define a class DatasetArguments using the dataclass decorator.
FinetunerArguments	Adapt transformers.TrainingArguments
RewardModelTunerArguments	Arguments for reward modeling.
EvaluatorArguments	Define a class EvaluatorArguments using the dataclass decorator. The class contains several optional
InferencerArguments	Define a class InferencerArguments using the dataclass decorator. The class contains several optional
RaftAlignerArguments	Define a class RaftAlignerArguments to configure raft aligner.
BenchmarkingArguments
DPOAlignerArguments	The arguments for the DPO training script.
DPOv2AlignerArguments	The arguments for the DPOv2 training script.
IterativeAlignerArguments	Arguments for iterative aligners.
IterativeDPOAlignerArguments	Arguments for iterative DPO aligners.
AutoArguments	Automatically choose arguments from FinetunerArguments or EvaluatorArguments.

Module Contents

lmflow.args.MODEL_CONFIG_CLASSES

lmflow.args.MODEL_TYPES

lmflow.args.logger

class lmflow.args.OptimizerNames

DUMMY = 'dummy'

ADABELIEF = 'adabelief'

ADABOUND = 'adabound'

LARS = 'lars'

LAMB = 'lamb'

ADAMAX = 'adamax'

NADAM = 'nadam'

RADAM = 'radam'

ADAMP = 'adamp'

SGDP = 'sgdp'

YOGI = 'yogi'

SOPHIA = 'sophia'

ADAN = 'adan'

ADAM = 'adam'

NOVOGRAD = 'novograd'

ADADELTA = 'adadelta'

ADAGRAD = 'adagrad'

ADAMW_SCHEDULE_FREE = 'adamw_schedule_free'

SGD_SCHEDULE_FREE = 'sgd_schedule_free'

class lmflow.args.ModelArguments

Define a class ModelArguments using the dataclass decorator. The class contains several optional parameters that can be used to configure a model.

model_name_or_pathstr

a string representing the path or name of a pretrained model checkpoint for weights initialization. If None, a model will be trained from scratch.

model_typestr

a string representing the type of model to use if training from scratch. If not provided, a pretrained model will be used.

config_overridesstr

a string representing the default config settings to override when training a model from scratch.

config_namestr

a string representing the name or path of the pretrained config to use, if different from the model_name_or_path.

tokenizer_namestr

a string representing the name or path of the pretrained tokenizer to use, if different from the model_name_or_path.

cache_dirstr

a string representing the path to the directory where pretrained models downloaded from huggingface.co will be stored.

use_fast_tokenizerbool

a boolean indicating whether to use a fast tokenizer (backed by the tokenizers library) or not.

model_revisionstr

a string representing the specific model version to use (can be a branch name, tag name, or commit id).

tokenOptional[str]

Necessary when accessing a private model/dataset.

torch_dtypestr

a string representing the dtype to load the model under. If auto is passed, the dtype will be automatically derived from the model’s weights.

use_ram_optimized_loadbool

a boolean indicating whether to use disk mapping when memory is not enough.

use_int8bool

a boolean indicating whether to load int8 quantization for inference.

load_in_4bitbool

whether to load the model in 4bit

model_max_lengthint

The maximum length of the model.

truncation_sidestr

The side on which the model should have truncation applied.

arch_typestr

Model architecture type.

padding_sidestr

The side on which the tokenizer should have padding applied.

eos_paddingbool

whether to pad with eos token instead of pad token.

ignore_bias_buffersbool

fix for DDP issues with LM bias/mask buffers - invalid scalar type,`inplace operation.

model_name_or_path: str | None

lora_model_path: str | None

model_type: str | None

config_overrides: str | None

arch_type: str | None

config_name: str | None

tokenizer_name: str | None

cache_dir: str | None

use_fast_tokenizer: bool

model_revision: str

token: str | None

trust_remote_code: bool

torch_dtype: str | None

use_lora: bool

use_qlora: bool

bits: int

quant_type: str

double_quant: bool

lora_r: int

lora_alpha: int

lora_target_modules: List[str]

lora_dropout: float

save_aggregated_lora: bool

use_ram_optimized_load: bool

use_flash_attention: bool

truncate_to_model_max_length: bool

do_rope_scaling: bool

rope_pi_ratio: int

rope_ntk_ratio: int

use_int8: bool

load_in_4bit: bool | None

model_max_length: int | None

truncation_side: str

padding_side: str

eos_padding: bool | None

ignore_bias_buffers: bool | None

__post_init__()

class lmflow.args.VisModelArguments

Bases: ModelArguments

Define a class ModelArguments using the dataclass decorator. The class contains several optional parameters that can be used to configure a model.

model_name_or_pathstr

a string representing the path or name of a pretrained model checkpoint for weights initialization. If None, a model will be trained from scratch.

model_typestr

a string representing the type of model to use if training from scratch. If not provided, a pretrained model will be used.

config_overridesstr

a string representing the default config settings to override when training a model from scratch.

config_namestr

a string representing the name or path of the pretrained config to use, if different from the model_name_or_path.

tokenizer_namestr

a string representing the name or path of the pretrained tokenizer to use, if different from the model_name_or_path.

cache_dirstr

a string representing the path to the directory where pretrained models downloaded from huggingface.co will be stored.

use_fast_tokenizerbool

a boolean indicating whether to use a fast tokenizer (backed by the tokenizers library) or not.

model_revisionstr

a string representing the specific model version to use (can be a branch name, tag name, or commit id).

tokenOptional[str]

Necessary when accessing a private model/dataset.

torch_dtypestr

a string representing the dtype to load the model under. If auto is passed, the dtype will be automatically derived from the model’s weights.

use_ram_optimized_loadbool

a boolean indicating whether to use disk mapping when memory is not enough.

use_int8bool

a boolean indicating whether to load int8 quantization for inference.

load_in_4bitbool

whether to load the model in 4bit

model_max_lengthint

The maximum length of the model.

truncation_sidestr

The side on which the model should have truncation applied.

arch_typestr

Model architecture type.

padding_sidestr

The side on which the tokenizer should have padding applied.

eos_paddingbool

whether to pad with eos token instead of pad token.

ignore_bias_buffersbool

fix for DDP issues with LM bias/mask buffers - invalid scalar type,`inplace operation.

low_resource: bool | None

custom_model: bool

pretrained_language_projection_path: str

custom_vision_model: bool

image_encoder_name_or_path: str | None

qformer_name_or_path: str | None

llm_model_name_or_path: str | None

use_prompt_cache: bool

prompt_cache_path: str | None

llava_loading: bool | None

with_qformer: bool | None

vision_select_layer: int | None

llava_pretrain_model_path: str | None

save_pretrain_model_path: str | None

class lmflow.args.DatasetArguments

Define a class DatasetArguments using the dataclass decorator. The class contains several optional parameters that can be used to configure a dataset for a language model.

dataset_pathstr

a string representing the path of the dataset to use.

dataset_namestr

a string representing the name of the dataset to use. The default value is “customized”.

is_custom_datasetbool

a boolean indicating whether to use custom data. The default value is False.

customized_cache_dirstr

a string representing the path to the directory where customized dataset caches will be stored.

dataset_config_namestr

a string representing the configuration name of the dataset to use (via the datasets library).

train_filestr

a string representing the path to the input training data file (a text file).

validation_filestr

a string representing the path to the input evaluation data file to evaluate the perplexity on (a text file).

max_train_samplesint

an integer indicating the maximum number of training examples to use for debugging or quicker training. If set, the training dataset will be truncated to this number.

max_eval_samples: int

an integer indicating the maximum number of evaluation examples to use for debugging or quicker training. If set, the evaluation dataset will be truncated to this number.

streamingbool

a boolean indicating whether to enable streaming mode.

block_size: int

an integer indicating the optional input sequence length after tokenization. The training dataset will be truncated in blocks of this size for training.

train_on_prompt: bool

a boolean indicating whether to train on prompt for conversation datasets such as ShareGPT.

conversation_template: str

a string representing the template for conversation datasets.

The class also includes some additional parameters that can be used to configure the dataset further, such as overwrite_cache, validation_split_percentage, preprocessing_num_workers, disable_group_texts, demo_example_in_prompt, explanation_in_prompt, keep_linebreaks, and prompt_structure.

The field function is used to set default values and provide help messages for each parameter. The Optional type hint is used to indicate that a parameter is optional. The metadata argument is used to provide additional information about each parameter, such as a help message.

dataset_path: str | None

dataset_name: str | None

is_custom_dataset: bool | None

customized_cache_dir: str | None

dataset_config_name: str | None

train_file: str | None

validation_file: str | None

max_train_samples: int | None

max_eval_samples: int | None

streaming: bool

block_size: int | None

overwrite_cache: bool

validation_split_percentage: int | None

preprocessing_num_workers: int | None

group_texts_batch_size: int

disable_group_texts: bool

keep_linebreaks: bool

test_file: str | None

train_on_prompt: bool

conversation_template: str | None

__post_init__()

class lmflow.args.MultiModalDatasetArguments

Bases: DatasetArguments

Define a class DatasetArguments using the dataclass decorator. The class contains several optional parameters that can be used to configure a dataset for a language model.

dataset_pathstr

a string representing the path of the dataset to use.

dataset_namestr

a string representing the name of the dataset to use. The default value is “customized”.

is_custom_datasetbool

a boolean indicating whether to use custom data. The default value is False.

customized_cache_dirstr

a string representing the path to the directory where customized dataset caches will be stored.

dataset_config_namestr

a string representing the configuration name of the dataset to use (via the datasets library).

train_filestr

a string representing the path to the input training data file (a text file).

validation_filestr

a string representing the path to the input evaluation data file to evaluate the perplexity on (a text file).

max_train_samplesint

an integer indicating the maximum number of training examples to use for debugging or quicker training. If set, the training dataset will be truncated to this number.

max_eval_samples: int

an integer indicating the maximum number of evaluation examples to use for debugging or quicker training. If set, the evaluation dataset will be truncated to this number.

streamingbool

a boolean indicating whether to enable streaming mode.

block_size: int

an integer indicating the optional input sequence length after tokenization. The training dataset will be truncated in blocks of this size for training.

train_on_prompt: bool

a boolean indicating whether to train on prompt for conversation datasets such as ShareGPT.

conversation_template: str

a string representing the template for conversation datasets.

The class also includes some additional parameters that can be used to configure the dataset further, such as overwrite_cache, validation_split_percentage, preprocessing_num_workers, disable_group_texts, demo_example_in_prompt, explanation_in_prompt, keep_linebreaks, and prompt_structure.

The field function is used to set default values and provide help messages for each parameter. The Optional type hint is used to indicate that a parameter is optional. The metadata argument is used to provide additional information about each parameter, such as a help message.

image_folder: str | None

image_aspect_ratio: str | None

is_multimodal: bool | None

use_image_start_end: bool | None

sep_style: str | None

class lmflow.args.FinetunerArguments

Bases: transformers.TrainingArguments

Adapt transformers.TrainingArguments

eval_dataset_path: str | None

remove_unused_columns: bool | None

finetune_part: str | None

save_language_projection: str | None

use_lisa: bool

lisa_activated_layers: int

lisa_interval_steps: int

lisa_layers_attribute: str

use_customized_optim: bool

customized_optim: str

customized_optim_args: str

optim_dummy_beta1: float

optim_dummy_beta2: float

optim_adam_beta1: float

optim_adam_beta2: float

optim_beta1: float

optim_beta2: float

optim_beta3: float

optim_momentum: float

optim_weight_decay: float

class lmflow.args.RewardModelTunerArguments

Bases: FinetunerArguments

Arguments for reward modeling.

class lmflow.args.EvaluatorArguments

Define a class EvaluatorArguments using the dataclass decorator. The class contains several optional parameters that can be used to configure a evaluator.

local_rankstr

For distributed training: local_rank

random_shuffle : bool

use_wandb : bool

random_seed : int, default = 1

output_dir : str, default = ‘./output_dir’,

mixed_precisionstr, choice from [“bf16”,”fp16”].

mixed precision mode, whether to use bf16 or fp16

deepspeed :

Enable deepspeed and pass the path to deepspeed json config file (e.g. ds_config.json) or an already loaded json file as a dict

temperaturefloat

An argument of model.generate in huggingface to control the diversity of generation.

repetition_penaltyfloat

An argument of model.generate in huggingface to penalize repetitions.

local_rank: int

random_shuffle: bool | None

use_wandb: bool | None

random_seed: int | None

output_dir: str | None

mixed_precision: str | None

deepspeed: str | None

answer_type: str | None

prompt_structure: str | None

evaluate_block_size: int | None

metric: str | None

inference_batch_size_per_device: int | None

use_accelerator_for_evaluator: bool

temperature: float

repetition_penalty: float

max_new_tokens: int

class lmflow.args.InferencerArguments

Define a class InferencerArguments using the dataclass decorator. The class contains several optional parameters that can be used to configure a inferencer.

local_rankstr

For distributed training: local_rank

random_seed : int, default = 1 inference_batch_size : int, default = 1 deepspeed :

Enable deepspeed and pass the path to deepspeed json config file (e.g. ds_config.json) or an already loaded json file as a dict

mixed_precisionstr, choice from [“bf16”,”fp16”].

mixed precision mode, whether to use bf16 or fp16

temperaturefloat

An argument of model.generate in huggingface to control the diversity of generation.

repetition_penaltyfloat

An argument of model.generate in huggingface to penalize repetitions.

use_beam_searchOptional[bool]

Whether to use beam search during inference, By default False.

num_output_sequencesOptional[int]

Number of output sequences to return for the given prompt, currently only used in vllm inference, By default 8.

top_pOptional[float]

top_p for sampling, By default 1.0.

top_kOptional[int]

top_k for sampling, By default -1 (no top_k).

additional_stop_token_idsOptional[List[int]]

the ids of the end of sentence tokens, By default [].

apply_chat_templateOptional[bool]

Whether to apply chat template, By default True.

save_resultsOptional[bool]

Whether to save inference results, By default False.

results_pathOptional[str]

The json file path of inference results, By default None.

enable_decode_inference_resultOptional[bool]

Whether to detokenize the inference results.

NOTE: For iterative align pipelines, whether to detokenize depends on the homogeneity of the policy model and the reward model (i.e., if they have the same tokenizer).

use_vllm: bool, optional

Whether to use VLLM for inference, By default False.

vllm_tensor_parallel_size: int, optional

The tensor parallel size for VLLM inference.

vllm_gpu_memory_utilization: float, optional

The GPU memory utilization for VLLM inference. The proportion of GPU memory (per GPU) to use for VLLM inference.

device: str

local_rank: int

inference_batch_size: int

vllm_inference_batch_size: int

temperature: float

repetition_penalty: float

max_new_tokens: int

random_seed: int | None

deepspeed: str | None

mixed_precision: str | None

do_sample: bool | None

use_accelerator: bool

use_beam_search: bool | None

num_output_sequences: int | None

top_p: float | None

top_k: int | None

additional_stop_token_ids: List[int] | None

apply_chat_template: bool | None

enable_decode_inference_result: bool | None

tensor_parallel_size: int | None

enable_distributed_inference: bool | None

distributed_inference_num_instances: int | None

use_vllm: bool

vllm_tensor_parallel_size: int | None

vllm_gpu_memory_utilization: float | None

save_results: bool | None

results_path: str | None

__post_init__()

class lmflow.args.RaftAlignerArguments

Bases: transformers.TrainingArguments

Define a class RaftAlignerArguments to configure raft aligner.

output_reward_path: str | None

output_min_length: int | None

output_max_length: int | None

num_raft_iteration: int | None

raft_batch_size: int | None

top_reward_percentage: float | None

inference_batch_size_per_device: int | None

collection_strategy: str | None

class lmflow.args.BenchmarkingArguments

dataset_name: str | None

lm_evaluation_metric: str | None

class lmflow.args.DPOAlignerArguments

The arguments for the DPO training script.

local_rank: int

beta: float | None

learning_rate: float | None

lr_scheduler_type: str | None

warmup_steps: int | None

weight_decay: float | None

optimizer_type: str | None

per_device_train_batch_size: int | None

per_device_eval_batch_size: int | None

gradient_accumulation_steps: int | None

gradient_checkpointing: bool | None

gradient_checkpointing_use_reentrant: bool | None

max_prompt_length: int | None

max_length: int | None

max_steps: int | None

logging_steps: int | None

save_steps: int | None

eval_steps: int | None

output_dir: str | None

log_freq: int | None

sanity_check: bool | None

report_to: str | None

seed: int | None

run_name: str | None

eval_dataset_path: str | None

class lmflow.args.DPOv2AlignerArguments

Bases: FinetunerArguments

The arguments for the DPOv2 training script.

random_seed: int | None

accelerate_config_file: str | None

margin_scale: float | None

sampling_paired_method: str | None

length_penalty: float | None

max_length: int | None

max_prompt_length: int | None

mask_prompt: bool | None

beta: float | None

loss_type: str | None

class lmflow.args.IterativeAlignerArguments

Bases: InferencerArguments

Arguments for iterative aligners.

dataset_path_list: List[str]

initial_iter_idx: int

class lmflow.args.IterativeDPOAlignerArguments

Bases: IterativeAlignerArguments, DPOv2AlignerArguments

Arguments for iterative DPO aligners.

output_dir: str | None

reward_model_inference_batch_size: int

reward_model_inference_block_size: int

do_response_generation: bool

do_scoring: bool

do_dpo_align: bool

lmflow.args.PIPELINE_ARGUMENT_MAPPING

class lmflow.args.AutoArguments

Automatically choose arguments from FinetunerArguments or EvaluatorArguments.

get_pipeline_args_class()