lmflow.models.hf_text_regression_model#
Attributes#
Classes#
Initializes a HFTextRegressionModel instance. |
Module Contents#
- class lmflow.models.hf_text_regression_model.HFTextRegressionModel(model_args: lmflow.args.ModelArguments, tune_strategy: str = 'normal', ds_config=None, device='gpu', use_accelerator=False, *args, **kwargs)[source]#
Bases:
lmflow.models.text_regression_model.TextRegressionModel
,lmflow.models.hf_model_mixin.HFModelMixin
,lmflow.models.interfaces.tunable.Tunable
Initializes a HFTextRegressionModel instance.
- Parameters:
- model_args
Model arguments such as model name, path, revision, etc.
- tune_strategystr or none, default=”normal”.
A string representing the dataset backend. Defaults to “huggingface”.
- ds_config
Deepspeed configuations.
- argsOptional.
Positional arguments.
- kwargsOptional.
Keyword arguments.
- tokenize(dataset: lmflow.datasets.dataset.Dataset, add_special_tokens=True, *args, **kwargs)[source]#
Tokenize the full dataset.
- Parameters:
- datasetlmflow.datasets.Dataset.
- argsOptional.
Positional arguments.
- kwargsOptional.
Keyword arguments.
- Returns:
- tokenized_datasets
The tokenized dataset, without any leading or trailing special tokens (normally they are Begin-Of-Sentence or End-Of-Sentence tokens).
- inference(inputs, release_gpu: bool = False, use_vllm: bool = False, **kwargs) List[float] | transformers.modeling_outputs.SequenceClassifierOutputWithPast [source]#
Perform generation process of the model.
- Parameters:
- inputs
The sequence used as a prompt for the generation or as model inputs to the model. When using vllm inference, this should be a string or a list of strings. When using normal inference, this should be a tensor.
- release_gpubool, optional
Whether to release the GPU resource after inference, by default False.
- use_vllmbool, optional
Whether to use VLLM for inference, by default False.
- kwargsOptional.
Keyword arguments.
- Returns:
- outputs
The generated sequence output
- __inference(inputs, **kwargs)[source]#
Perform generation process of the model.
- Parameters:
- inputs
The tokenized sequence used as a prompt for the generation or as model inputs to the model.
- kwargsOptional.
Keyword arguments.
- Returns:
- outputs
The generated sequence output
- abstract __vllm_inference(inputs: str | List[str], sampling_params: vllm.SamplingParams | None = None, **kwargs) List[List[str]] | List[List[List[int]]] [source]#
Perform VLLM inference process of the model.
- Parameters:
- inputsUnion[str, List[str]]
Prompt(s), string or a list of strings.
- sampling_paramsOptional[SamplingParams], optional
vllm SamplingParams object, by default None.
- Returns:
- prepare_inputs_for_inference(dataset: lmflow.datasets.dataset.Dataset, enable_distributed_inference: bool = False, use_vllm: bool = False, **kwargs) lmflow.datasets.dataset.Dataset | ray.data.Dataset [source]#
- static postprocess_inference_outputs(dataset: lmflow.datasets.dataset.Dataset, scores: List[float] | List[List[float]])[source]#
- static postprocess_distributed_inference_outputs(dataset: lmflow.datasets.dataset.Dataset, inference_result: List[lmflow.utils.data_utils.RewardModelInferenceResultWithInput])[source]#