Supported Conversation Template#
ChatGLM-3#
With a system message
[gMASK]sop<|system|>\n {{system_message}}<|user|>\n {{user_message_0}}
Without a system message
[gMASK]sop<|user|>\n {{user_message_0}}
A complete conversation
[gMASK]sop<|system|>\n {{system_message}}<|user|>\n {{user_message_0}}<|assistant|>\n {{assistant_reply_0}}
Multiple rounds
[gMASK]sop<|system|>\n {{system_message}}<|user|>\n {{user_message_0}}<|assistant|>\n {{assistant_reply_0}}<|user|>\n {{user_message_1}}<|assistant|>\n {{assistant_reply_1}}
jinja template
[Reference]
{% for message in messages %}{% if loop.first %}[gMASK]sop<|{{ message['role'] }}|>\n {{ message['content'] }}{% else %}<|{{ message['role'] }}|>\n {{ message['content'] }}{% endif %}{% endfor %}{% if add_generation_prompt %}<|assistant|>{% endif %}
Filled Example
[gMASK]sop<|system|>\n You are a chatbot developed by LMFlow team.<|user|>\n Who are you?<|assistant|>\n I am a chatbot developed by LMFlow team.<|user|>\n How old are you?<|assistant|>\n I don't age like humans do. I exist as a piece of software, so I don't have a concept of age in the traditional sense.
ChatML#
With a system message
<|im_start|>system\n{{system_message}}<|im_end|>\n<|im_start|>user\n{{user_message_0}}<|im_end|>\n
Without a system message
<|im_start|>user\n{{user_message_0}}<|im_end|>\n
A complete conversation
<|im_start|>system\n{{system_message}}<|im_end|>\n<|im_start|>user\n{{user_message_0}}<|im_end|>\n<|im_start|>assistant\n{{assistant_reply_0}}<|im_end|>\n
Multiple rounds
<|im_start|>system\n{{system_message}}<|im_end|>\n<|im_start|>user\n{{user_message_0}}<|im_end|>\n<|im_start|>assistant\n{{assistant_reply_0}}<|im_end|>\n<|im_start|>user\n{{user_message_1}}<|im_end|>\n<|im_start|>assistant\n{{assistant_reply_1}}<|im_end|>\n
jinja template
[Reference]
{% for message in messages %}{{'<|im_start|>' + message['role'] + '\n' + message['content'] + '<|im_end|>' + '\n'}}{% endfor %}{% if add_generation_prompt %}{{ '<|im_start|>assistant\n' }}{% endif %}
Filled Example
<|im_start|>system\nYou are a chatbot developed by LMFlow team.<|im_end|>\n<|im_start|>user\nWho are you?<|im_end|>\n<|im_start|>assistant\nI am a chatbot developed by LMFlow team.<|im_end|>\n<|im_start|>user\nHow old are you?<|im_end|>\n<|im_start|>assistant\nI don't age like humans do. I exist as a piece of software, so I don't have a concept of age in the traditional sense.<|im_end|>\n
DeepSeek#
With a system message
<｜begin▁of▁sentence｜>{{system_message}}\n\nUser: {{user_message_0}}\n\n
Without a system message
<｜begin▁of▁sentence｜>User: {{user_message_0}}\n\n
A complete conversation
<｜begin▁of▁sentence｜>{{system_message}}\n\nUser: {{user_message_0}}\n\nAssistant: {{assistant_reply_0}}<｜end▁of▁sentence｜>
Multiple rounds
<｜begin▁of▁sentence｜>{{system_message}}\n\nUser: {{user_message_0}}\n\nAssistant: {{assistant_reply_0}}<｜end▁of▁sentence｜>User: {{user_message_1}}\n\nAssistant: {{assistant_reply_1}}<｜end▁of▁sentence｜>
jinja template
[Reference]
{% if not add_generation_prompt is defined %}{% set add_generation_prompt = false %}{% endif %}{{ bos_token }}{% for message in messages %}{% if message['role'] == 'user' %}{{ 'User: ' + message['content'] + '\n\n' }}{% elif message['role'] == 'assistant' %}{{ 'Assistant: ' + message['content'] + eos_token }}{% elif message['role'] == 'system' %}{{ message['content'] + '\n\n' }}{% endif %}{% endfor %}{% if add_generation_prompt %}{{ 'Assistant:' }}{% endif %}
Filled Example
<｜begin▁of▁sentence｜>You are a chatbot developed by LMFlow team.\n\nUser: Who are you?\n\nAssistant: I am a chatbot developed by LMFlow team.<｜end▁of▁sentence｜>User: How old are you?\n\nAssistant: I don't age like humans do. I exist as a piece of software, so I don't have a concept of age in the traditional sense.<｜end▁of▁sentence｜>
Gemma#
With a system message
NOTICE
As of now, Gemma does not support system messages officially. ConversationTemplate will add your system messages right after the bos token and before the user message without any special formatting. For more details, please refer to the official template.
<bos>{{system_message}}<start_of_turn>user\n{{user_message_0}}<end_of_turn>\n
Without a system message
<bos><start_of_turn>user\n{{user_message_0}}<end_of_turn>\n
A complete conversation
<bos>{{system_message}}<start_of_turn>user\n{{user_message_0}}<end_of_turn>\n<start_of_turn>model\n{{assistant_reply_0}}<end_of_turn>\n
Multiple rounds
<bos>{{system_message}}<start_of_turn>user\n{{user_message_0}}<end_of_turn>\n<start_of_turn>model\n{{assistant_reply_0}}<end_of_turn>\n<start_of_turn>user\n{{user_message_1}}<end_of_turn>\n<start_of_turn>model\n{{assistant_reply_1}}<end_of_turn>\n
jinja template
[Reference]
{{ bos_token }}{% if messages[0]['role'] == 'system' %}{{ raise_exception('System role not supported') }}{% endif %}{% for message in messages %}{% if (message['role'] == 'user') != (loop.index0 % 2 == 0) %}{{ raise_exception('Conversation roles must alternate user/assistant/user/assistant/...') }}{% endif %}{% if (message['role'] == 'assistant') %}{% set role = 'model' %}{% else %}{% set role = message['role'] %}{% endif %}{{ '<start_of_turn>' + role + '\n' + message['content'] | trim + '<end_of_turn>\n' }}{% endfor %}{% if add_generation_prompt %}{{'<start_of_turn>model\n'}}{% endif %}
Filled Example
<bos>You are a chatbot developed by LMFlow team.<start_of_turn>user\nWho are you?<end_of_turn>\n<start_of_turn>model\nI am a chatbot developed by LMFlow team.<end_of_turn>\n<start_of_turn>user\nHow old are you?<end_of_turn>\n<start_of_turn>model\nI don't age like humans do. I exist as a piece of software, so I don't have a concept of age in the traditional sense.<end_of_turn>\n
InternLM2#
With a system message
<s><|im_start|>system\n{{system_message}}<|im_end|>\n<|im_start|>user\n{{user_message_0}}<|im_end|>\n
Without a system message
<s><|im_start|>user\n{{user_message_0}}<|im_end|>\n
A complete conversation
<s><|im_start|>system\n{{system_message}}<|im_end|>\n<|im_start|>user\n{{user_message_0}}<|im_end|>\n<|im_start|>assistant\n{{assistant_reply_0}}<|im_end|>\n
Multiple rounds
<s><|im_start|>system\n{{system_message}}<|im_end|>\n<|im_start|>user\n{{user_message_0}}<|im_end|>\n<|im_start|>assistant\n{{assistant_reply_0}}<|im_end|>\n<|im_start|>user\n{{user_message_1}}<|im_end|>\n<|im_start|>assistant\n{{assistant_reply_1}}<|im_end|>\n
jinja template
[Reference]
{{ bos_token }}{% for message in messages %}{{'<|im_start|>' + message['role'] + '\n' + message['content'] + '<|im_end|>' + '\n'}}{% endfor %}{% if add_generation_prompt %}{{ '<|im_start|>assistant\n' }}{% endif %}
Filled Example
<s><|im_start|>system\nYou are a chatbot developed by LMFlow team.<|im_end|>\n<|im_start|>user\nWho are you?<|im_end|>\n<|im_start|>assistant\nI am a chatbot developed by LMFlow team.<|im_end|>\n<|im_start|>user\nHow old are you?<|im_end|>\n<|im_start|>assistant\nI don't age like humans do. I exist as a piece of software, so I don't have a concept of age in the traditional sense.<|im_end|>\n
Llama-2#
With a system message
<s>[INST] <<SYS>>\n{{system_message}}\n<</SYS>>\n\n{{user_message_0}} [/INST]
Without a system message
<s>[INST] {{user_message_0}} [/INST]
A complete conversation
<s>[INST] <<SYS>>\n{{system_message}}\n<</SYS>>\n\n{{user_message_0}} [/INST] {{assistant_reply_0}}</s>
Multiple rounds
<s>[INST] <<SYS>>\n{{system_message}}\n<</SYS>>\n\n{{user_message_0}} [/INST] {{assistant_reply_0}}</s><s>[INST] {{user_message_1}} [/INST] {{assistant_reply_1}}</s>
jinja template
[Reference]
{% if messages[0]['role'] == 'system' %}{% set loop_messages = messages[1:] %}{% set system_message = messages[0]['content'] %}{% else %}{% set loop_messages = messages %}{% set system_message = false %}{% endif %}{% for message in loop_messages %}{% if (message['role'] == 'user') != (loop.index0 % 2 == 0) %}{{ raise_exception('Conversation roles must alternate user/assistant/user/assistant/...') }}{% endif %}{% if loop.index0 == 0 and system_message != false %}{% set content = '<<SYS>>\\n' + system_message + '\\n<</SYS>>\\n\\n' + message['content'] %}{% else %}{% set content = message['content'] %}{% endif %}{% if message['role'] == 'user' %}{{ bos_token + '[INST] ' + content.strip() + ' [/INST]' }}{% elif message['role'] == 'assistant' %}{{ ' ' + content.strip() + ' ' + eos_token }}{% endif %}{% endfor %}
Filled Example
<s>[INST] <<SYS>>\nYou are a chatbot developed by LMFlow team.\n<</SYS>>\n\nWho are you? [/INST] I am a chatbot developed by LMFlow team.</s><s>[INST] How old are you? [/INST] I don't age like humans do. I exist as a piece of software, so I don't have a concept of age in the traditional sense.</s>
Llama-3#
With a system message
<|begin_of_text|><|start_header_id|>system<|end_header_id|>\n\n{{system_message}}<|eot_id|><|start_header_id|>user<|end_header_id|>\n\n{{user_message_0}}<|eot_id|>
Without a system message
<|begin_of_text|><|start_header_id|>user<|end_header_id|>\n\n{{user_message_0}}<|eot_id|>
A complete conversation
<|begin_of_text|><|start_header_id|>system<|end_header_id|>\n\n{{system_message}}<|eot_id|><|start_header_id|>user<|end_header_id|>\n\n{{user_message_0}}<|eot_id|><|start_header_id|>assistant<|end_header_id|>\n\n{{assistant_reply_0}}<|eot_id|>
Multiple rounds
<|begin_of_text|><|start_header_id|>system<|end_header_id|>\n\n{{system_message}}<|eot_id|><|start_header_id|>user<|end_header_id|>\n\n{{user_message_0}}<|eot_id|><|start_header_id|>assistant<|end_header_id|>\n\n{{assistant_reply_0}}<|eot_id|><|start_header_id|>user<|end_header_id|>\n\n{{user_message_1}}<|eot_id|><|start_header_id|>assistant<|end_header_id|>\n\n{{assistant_reply_1}}<|eot_id|>
jinja template
[Reference]
{% set loop_messages = messages %}{% for message in loop_messages %}{% set content = '<|start_header_id|>' + message['role'] + '<|end_header_id|>\n\n'+ message['content'] | trim + '<|eot_id|>' %}{% if loop.index0 == 0 %}{% set content = bos_token + content %}{% endif %}{{ content }}{% endfor %}{{ '<|start_header_id|>assistant<|end_header_id|>\n\n' }}
Filled Example
<|begin_of_text|><|start_header_id|>system<|end_header_id|>\n\nYou are a chatbot developed by LMFlow team.<|eot_id|><|start_header_id|>user<|end_header_id|>\n\nWho are you?<|eot_id|><|start_header_id|>assistant<|end_header_id|>\n\nI am a chatbot developed by LMFlow team.<|eot_id|><|start_header_id|>user<|end_header_id|>\n\nHow old are you?<|eot_id|><|start_header_id|>assistant<|end_header_id|>\n\nI don't age like humans do. I exist as a piece of software, so I don't have a concept of age in the traditional sense.<|eot_id|>
Mixtral 8x22B#
Work in Progress
This template is not preseted in LMFlow currently. We are working on it and will update it soon.
NOTICE
The conversation template for Mixtral 8x22B is slightly different from the template for Mixtral 8x7B.
jinja template
[Reference]
{{bos_token}}{% for message in messages %}{% if (message['role'] == 'user') != (loop.index0 % 2 == 0) %}{{ raise_exception('Conversation roles must alternate user/assistant/user/assistant/...') }}{% endif %}{% if message['role'] == 'user' %}{{ ' [INST] ' + message['content'] + ' [/INST]' }}{% elif message['role'] == 'assistant' %}{{ ' ' + message['content'] + ' ' + eos_token}}{% else %}{{ raise_exception('Only user and assistant roles are supported!') }}{% endif %}{% endfor %}
Mixtral 8x7B#
Work in Progress
This template is not preseted in LMFlow currently. We are working on it and will update it soon.
NOTICE
The conversation template for Mixtral 8x7B is slightly different from the template for Mixtral 8x22B.
jinja template
[Reference]
{{ bos_token }}{% for message in messages %}{% if (message['role'] == 'user') != (loop.index0 % 2 == 0) %}{{ raise_exception('Conversation roles must alternate user/assistant/user/assistant/...') }}{% endif %}{% if message['role'] == 'user' %}{{ '[INST] ' + message['content'] + ' [/INST]' }}{% elif message['role'] == 'assistant' %}{{ message['content'] + eos_token}}{% else %}{{ raise_exception('Only user and assistant roles are supported!') }}{% endif %}{% endfor %}
Phi-3#
With a system message
<s><|system|>\n{{system_message}}<|end|>\n<|user|>\n{{user_message_0}}<|end|>\n<|endoftext|>
Without a system message
<s><|user|>\n{{user_message_0}}<|end|>\n<|endoftext|>
A complete conversation
<s><|system|>\n{{system_message}}<|end|>\n<|user|>\n{{user_message_0}}<|end|>\n<|assistant|>\n{{assistant_reply_0}}<|end|>\n<|endoftext|>
Multiple rounds
<s><|system|>\n{{system_message}}<|end|>\n<|user|>\n{{user_message_0}}<|end|>\n<|assistant|>\n{{assistant_reply_0}}<|end|>\n<|user|>\n{{user_message_1}}<|end|>\n<|assistant|>\n{{assistant_reply_1}}<|end|>\n<|endoftext|>
jinja template
[Reference]
{{ bos_token }}{% for message in messages %}{{'<|' + message['role'] + '|>' + '\n' + message['content'] + '<|end|>\n' }}{% endfor %}{% if add_generation_prompt %}{{ '<|assistant|>\n' }}{% else %}{{ eos_token }}{% endif %}
Filled Example
<s><|system|>\nYou are a chatbot developed by LMFlow team.<|end|>\n<|user|>\nWho are you?<|end|>\n<|assistant|>\nI am a chatbot developed by LMFlow team.<|end|>\n<|user|>\nHow old are you?<|end|>\n<|assistant|>\nI don't age like humans do. I exist as a piece of software, so I don't have a concept of age in the traditional sense.<|end|>\n<|endoftext|>
Qwen-2#
(Also Qwen-1.5)
With a system message
<|im_start|>system\n{{system_message}}<|im_end|>\n<|im_start|>user\n{{user_message_0}}<|im_end|>\n
Without a system message
<|im_start|>user\n{{user_message_0}}<|im_end|>\n
A complete conversation
<|im_start|>system\n{{system_message}}<|im_end|>\n<|im_start|>user\n{{user_message_0}}<|im_end|>\n<|im_start|>assistant\n{{assistant_reply_0}}<|im_end|>\n
Multiple rounds
<|im_start|>system\n{{system_message}}<|im_end|>\n<|im_start|>user\n{{user_message_0}}<|im_end|>\n<|im_start|>assistant\n{{assistant_reply_0}}<|im_end|>\n<|im_start|>user\n{{user_message_1}}<|im_end|>\n<|im_start|>assistant\n{{assistant_reply_1}}<|im_end|>\n
jinja template
[Reference]
{% for message in messages %}{% if loop.first and messages[0]['role'] != 'system' %}{{ '<|im_start|>system\nYou are a helpful assistant<|im_end|>\n' }}{% endif %}{{'<|im_start|>' + message['role'] + '\n' + message['content'] + '<|im_end|>' + '\n'}}{% endfor %}{% if add_generation_prompt %}{{ '<|im_start|>assistant\n' }}{% endif %}
Filled Example
<|im_start|>system\nYou are a chatbot developed by LMFlow team.<|im_end|>\n<|im_start|>user\nWho are you?<|im_end|>\n<|im_start|>assistant\nI am a chatbot developed by LMFlow team.<|im_end|>\n<|im_start|>user\nHow old are you?<|im_end|>\n<|im_start|>assistant\nI don't age like humans do. I exist as a piece of software, so I don't have a concept of age in the traditional sense.<|im_end|>\n
Yi#
With a system message
<|im_start|>system\n{{system_message}}<|im_end|>\n<|im_start|>user\n{{user_message_0}}<|im_end|>\n
Without a system message
<|im_start|>user\n{{user_message_0}}<|im_end|>\n
A complete conversation
<|im_start|>system\n{{system_message}}<|im_end|>\n<|im_start|>user\n{{user_message_0}}<|im_end|>\n<|im_start|>assistant\n{{assistant_reply_0}}<|im_end|>\n
Multiple rounds
<|im_start|>system\n{{system_message}}<|im_end|>\n<|im_start|>user\n{{user_message_0}}<|im_end|>\n<|im_start|>assistant\n{{assistant_reply_0}}<|im_end|>\n<|im_start|>user\n{{user_message_1}}<|im_end|>\n<|im_start|>assistant\n{{assistant_reply_1}}<|im_end|>\n
jinja template
[Reference]
{% if not add_generation_prompt is defined %}{% set add_generation_prompt = false %}{% endif %}{% for message in messages %}{{'<|im_start|>' + message['role'] + '\n' + message['content'] + '<|im_end|>' + '\n'}}{% endfor %}{% if add_generation_prompt %}{{ '<|im_start|>assistant\n' }}{% endif %}
Filled Example
<|im_start|>system\nYou are a chatbot developed by LMFlow team.<|im_end|>\n<|im_start|>user\nWho are you?<|im_end|>\n<|im_start|>assistant\nI am a chatbot developed by LMFlow team.<|im_end|>\n<|im_start|>user\nHow old are you?<|im_end|>\n<|im_start|>assistant\nI don't age like humans do. I exist as a piece of software, so I don't have a concept of age in the traditional sense.<|im_end|>\n
Yi-1.5#
With a system message
{{system_message}}<|im_start|>user\n{{user_message_0}}<|im_end|>\n
Without a system message
<|im_start|>user\n{{user_message_0}}<|im_end|>\n
A complete conversation
{{system_message}}<|im_start|>user\n{{user_message_0}}<|im_end|>\n<|im_start|>assistant\n{{assistant_reply_0}}<|im_end|>\n
Multiple rounds
{{system_message}}<|im_start|>user\n{{user_message_0}}<|im_end|>\n<|im_start|>assistant\n{{assistant_reply_0}}<|im_end|>\n<|im_start|>user\n{{user_message_1}}<|im_end|>\n<|im_start|>assistant\n{{assistant_reply_1}}<|im_end|>\n
jinja template
[Reference]
{% if messages[0]['role'] == 'system' %}{% set system_message = messages[0]['content'] %}{% endif %}{% if system_message is defined %}{{ system_message }}{% endif %}{% for message in messages %}{% set content = message['content'] %}{% if message['role'] == 'user' %}{{ '<|im_start|>user\\n' + content + '<|im_end|>\\n<|im_start|>assistant\\n' }}{% elif message['role'] == 'assistant' %}{{ content + '<|im_end|>' + '\\n' }}{% endif %}{% endfor %}
Filled Example
You are a chatbot developed by LMFlow team.<|im_start|>user\nWho are you?<|im_end|>\n<|im_start|>assistant\nI am a chatbot developed by LMFlow team.<|im_end|>\n<|im_start|>user\nHow old are you?<|im_end|>\n<|im_start|>assistant\nI don't age like humans do. I exist as a piece of software, so I don't have a concept of age in the traditional sense.<|im_end|>\n
Zephyr#
With a system message
<|system|>\n{{system_message}}</s>\n<|user|>\n{{user_message_0}}</s>\n
Without a system message
<|user|>\n{{user_message_0}}</s>\n
A complete conversation
<|system|>\n{{system_message}}</s>\n<|user|>\n{{user_message_0}}</s>\n<|assistant|>\n{{assistant_reply_0}}</s>\n
Multiple rounds
<|system|>\n{{system_message}}</s>\n<|user|>\n{{user_message_0}}</s>\n<|assistant|>\n{{assistant_reply_0}}</s>\n<|user|>\n{{user_message_1}}</s>\n<|assistant|>\n{{assistant_reply_1}}</s>\n
jinja template
[Reference]
{% for message in messages %}\n{% if message['role'] == 'user' %}\n{{ '<|user|>\n' + message['content'] + eos_token }}\n{% elif message['role'] == 'system' %}\n{{ '<|system|>\n' + message['content'] + eos_token }}\n{% elif message['role'] == 'assistant' %}\n{{ '<|assistant|>\n' + message['content'] + eos_token }}\n{% endif %}\n{% if loop.last and add_generation_prompt %}\n{{ '<|assistant|>' }}\n{% endif %}\n{% endfor %}
Filled Example
<|system|>\nYou are a chatbot developed by LMFlow team.</s>\n<|user|>\nWho are you?</s>\n<|assistant|>\nI am a chatbot developed by LMFlow team.</s>\n<|user|>\nHow old are you?</s>\n<|assistant|>\nI don't age like humans do. I exist as a piece of software, so I don't have a concept of age in the traditional sense.</s>\n