lmflow.pipeline.sglang_inferencer
=================================

.. py:module:: lmflow.pipeline.sglang_inferencer


Attributes
----------

.. autoapisummary::

   lmflow.pipeline.sglang_inferencer.logger


Classes
-------

.. autoapisummary::

   lmflow.pipeline.sglang_inferencer.SGLangInferencer


Module Contents
---------------

.. py:data:: logger

.. py:class:: SGLangInferencer(model_args: lmflow.args.ModelArguments, data_args: lmflow.args.DatasetArguments, inferencer_args: lmflow.args.InferencerArguments)

   Bases: :py:obj:`lmflow.pipeline.base_pipeline.BasePipeline`


   .. py:attribute:: model_args


   .. py:attribute:: data_args


   .. py:attribute:: inferencer_args


   .. py:attribute:: eos_token_id


   .. py:attribute:: sampling_params


   .. py:method:: _parse_args_to_sampling_params(inference_args: lmflow.args.InferencerArguments) -> dict


   .. py:method:: inference(model: lmflow.models.hf_decoder_model.HFDecoderModel, dataset: lmflow.datasets.Dataset, release_gpu: bool = False, inference_args: Optional[lmflow.args.InferencerArguments] = None)


   .. py:method:: save_inference_results(outputs: Union[list[list[str]], list[list[list[int]]]], save_file_path: str)


   .. py:method:: load_inference_results(results_path: str) -> Union[list[list[str]], list[list[list[int]]]]