about overview
all materials
Updated August 2026
tutorial · preprint · 2026

Using Customised LLM-Based Chatbots
in Psychological Research

This tutorial is designed for researchers interested in building customised LLM-based chatbots for psychological research.

Use this website to find step-by-step deployment instructions for Desktop, Telegram, and Qualtrics, download code templates,
example chatbot documentation, and review key security considerations for responsible implementation.

Cite

Hu, M., Lau, G. R., Goh, A. Y. H., Cox, S., Tay, L., & Hartanto, A. (2026). Using Customised LLM-based Chatbots in Psychological Research: A Practical Tutorial. https://doi.org/10.31234/osf.io/kf96p_v1

Materials
Desktop Implementation
Run the chatbot on your desktop using Python.
.zip
Telegram Implementation
Deploy the chatbot through Telegram.
.zip
Qualtrics Integration
Embed the chatbot within a Qualtrics survey.
.zip
Chatbot Documentation Template
Template for documenting chatbot prompts and implementations.
.pdf
Team

Meet the researchers behind the tutorial

Meilan Hu
Meilan Hu lead
Linkedin Scholar
GL
Gabriel R. Lau
Linkedin Scholar
AG
Adalia Y. H. Goh
Linkedin Scholar
SC
Samuel Rhys Cox
Linkedin Scholar
LT
Louis Tay
Linkedin Scholar
AH
Andree Hartanto
Scholar
deployment · desktop

Desktop

Run the chatbot on your own computer using a programming language such as Python and a command-line interface such as Terminal or PowerShell.

See Step 5 of the tutorial for more
required files
1
system_prompt.txt
Defines the chatbot's persona, instructions, and behavioural constraints.
2
.env
Stores your API key securely — never hardcode it into your main script.
3
main.py
Loads the prompt and key, calls the model, and manages the conversation loop.
4
requirements.txt
Lists external packages (e.g. openai, python-dotenv) needed to run the chatbot.
run the chatbot
1
Create virtual environment
python3 -m venv .venv
Creates an isolated Python environment for the project.
2
Activate it
source .venv/bin/activate
Activates the virtual environment.
3
Install packages
pip install -r requirements.txt
Installs the external packages listed in requirements.txt.
4
Run the chatbot
python3 main.py
Starts the chatbot.
1
Create virtual environment
python -m venv .venv
Creates an isolated Python environment for the project.
2
Activate it
.venv\Scripts\Activate.ps1
Activates the virtual environment.
3
Install packages
pip install -r requirements.txt
Installs the external packages listed in requirements.txt.
4
Run the chatbot
python main.py
Starts the chatbot.
download
Desktop Implementation
Run the chatbot on your desktop using Python
.zip
deployment · telegram

Telegram

Connect an LLM-based chatbot to Telegram so participants can interact with it through a familiar messaging interface.

See Step 5 of the tutorial for more
1
Create a Telegram bot
Obtain a Telegram Bot API token through @BotFather.
2
Configure the chatbot
In a visual or low-code platform such as Coze or Botpress, specify the chatbot’s system prompt, selected model, and behavioural settings.
3
Connect the chatbot to Telegram
Enter the Telegram Bot API token in the platform’s integration or channel settings and publish the chatbot to Telegram.

This example is based on Make.com, a no-code webhook automation service.

1
Create a Telegram bot
Obtain a Telegram Bot API token through @BotFather.
2
Create the automated workflow
Using a webhook-based automation service such as Make.com, configure a workflow that receives Telegram messages,
sends them to the selected model provider, and returns the generated response to the participant.
3
Configure the required components
Provide the Telegram Bot API token, model-provider API key, and system prompt.
1
Prepare the chatbot files
  • system_prompt.txt — chatbot role and behavioural instructions
  • .env — Telegram Bot API token and model-provider API key
  • main.py — receives Telegram messages, calls the LLM, and sends responses back
  • requirements.txt — required packages such as python-telegram-bot and python-dotenv
2
Connect the chatbot to Telegram
The bot script receives participant messages from Telegram, sends them to the selected model provider, and returns the generated response to the participant.
3
Choose how to run the chatbot
On your computer — Run the chatbot directly from your own computer. The script must remain running for the bot to receive and respond to Telegram messages.
On a server — Host the chatbot on a server so it can remain available without requiring your own computer to stay on.
download
Telegram Implementation
Deploy the chatbot through Telegram.
.zip
deployment · qualtrics

Qualtrics

Embed an LLM-based chatbot within a Qualtrics survey so conversational and survey data can be collected within the same study workflow.

See Step 5 of the tutorial for more
direct integration
1
Deploy an intermediary server
Use a backend server to manage communication between Qualtrics and the model provider. The server receives participant messages, calls the model API, and returns the generated response while keeping the API key inaccessible to participants.
2
Add the chatbot interface in Qualtrics
Create a Text/Graphic question, switch to HTML View, and insert the provided HTML code for the chatbot interface. Add the accompanying JavaScript code to capture participant messages and send them to the intermediary server.
3
Connect to the server
Set the ENDPOINT in the JavaScript to the deployed intermediary-server URL. Participant messages and the relevant model settings are then sent to this endpoint for processing.
4
Test the interaction
Preview the survey and confirm that participant messages reach the server, model responses are returned correctly, and the chatbot interface updates as intended.
participant data and survey flow
1
Participant identification
Qualtrics can use its automatically generated 'ResponseID', or a study-specific participant ID can be stored as 'Embedded Data'.
2
Condition assignment
For studies with multiple conditions, the 'Survey Flow Randomizer' can assign conditions and store the assigned condition as 'Embedded Data'.
3
Conversation logging
Chatbot conversations are not automatically exported as structured transcripts. If transcripts are needed, they may be captured separately and linked to the participant or condition data.
alternative: iframe embedding

Instead of building the full chatbot interface directly within Qualtrics, researchers can host the chatbot separately and display it inside the survey using an iFrame.

<iframe
  src="CHATBOT_URL"
  width="100%"
  height="600"
  title="Chatbot interface">
</iframe>
download
Qualtrics Integration
Embed the chatbot within a Qualtrics survey.
.zip
resources

Chatbot Design Guide

A comprehensive guide to designing and testing LLM-based chatbots in psychological research.

System Prompt Structure

The system prompt serves as a key component for guiding the chatbot's behaviour and interaction style.

See Step 2 of the tutorial for more
1
Define the chatbot's role
Start by specifying the chatbot's purpose within the research study.
2
Define the chatbot's persona
Define the character or personality that the chatbot should maintain throughout the conversation.
3
Set the interaction style
Specify how the chatbot should communicate with participants.
4
Specify instructions and behavioural boundaries
State clearly what the chatbot should and should not do.
Prompting Strategies

Beyond specifying the content of the system prompt, researchers may also employ different prompting strategies to
further refine the behaviour of the chatbot. Examples include:

See Step 2 of the tutorial for more
1
Few-shot prompting
Provide example user inputs and desired chatbot responses to demonstrate the intended behaviour.
2
Step-by-step prompting
Structure complex reasoning or decision tasks into a sequence of intermediate stages.
3
LLM-as-a-judge workflow
In API-based implementations, one or more evaluator models assess a draft response against predefined criteria.
Testing and Refinement

A well-designed prompt and carefully chosen settings do not ensure that the chatbot will behave as intended in actual interactions.
Researchers should test, evaluate, and iteratively refine the chatbot before deployment.

See Step 3 and 4 of the tutorial for more
1
Test a range of interactions
Researchers should evaluate and iteratively refine the chatbot's behavior before deployment.
2
Evaluate intended behavior
Evaluate whether the chatbot reliably performs its intended function using criteria appropriate to the study.
3
Refine
Refine the chatbot based on testing and evaluation findings, and retest after each modification until its behaviour is consistent with its intended role
resources

Chatbot Documentation

Document the chatbot’s design and implementation to support transparent and reproducible research.

1
What to document?
Document the chatbot’s conversational design, technical setup, and interaction flow to support transparency and reproducibility.
2
Why document this?
Differences in model version, API settings, or deployment environment may affect chatbot responses and therefore reproducibility.
3
How to document?
Use the provided template to record the chatbot’s design and implementation details. Include the system prompt, model settings,
and any other relevant information.
Summary of What to Document
See Step 6 of the tutorial for more
Design of Chatbot
  • Chatbot role
  • System prompt
  • Conversational stages
  • Example exchanges
  • Exit strategy
Implementation Details
  • Model and version
  • API settings
  • Safety procedures
  • Conversation-history handling
  • Deployment platform
  • Data-handling procedures
  • Code
download
Chatbot Documentation Template
Template for documenting the design and implementation of your chatbot.
.pdf
resources

Security Considerations

Practical guidance for securely deploying LLM-based chatbots, covering API keys, participant data, chatbot safety, and API usage considerations.

API key handling
1
Do not hardcode your API key
Do not paste your API key directly into your script. Anyone with access to your code, including via a public repository,
will be able to use your key and incur charges on your account. Always load keys from environment variables or a .env file.
2
Use a .env file
Store your key in a file named .env in your project root:
API_KEY=sk-your-key-here
Load it in Python using the python-dotenv package. Your script reads the key without it ever appearing in your code.
3
Add .env to .gitignore
Create a file named .gitignore in your project root and add the following lines to prevent your key from being accidentally committed to your repository:

.env
__pycache__/
*.pyc
.venv/
4
Rotate keys if exposed
If you accidentally commit a key to a public repository, assume that the key is compromised.
Immediately go to your API provider's dashboard, revoke the exposed key, and generate a new one.
data handling & privacy
1
Consider where the participant data is collected and stored
Consider where participant data are transmitted, what information is sent to external model providers, and whether the chosen setup is appropriate for the study.
2
Plan how data are stored and retained
Document how chatbot interactions are logged and stored, how long the data are retained, and the data-handling procedures used in the study.
3
Consider privacy and data-protection requirements
When chatbot conversations may contain personal or sensitive information, researchers should consider applicable institutional ethics requirements, data-protection frameworks, and the policies of any third-party providers involved.
chatbot safety
1
Define behavioural boundaries
Specify what the chatbot should and should not do, particularly when participants may discuss sensitive or distressing topics.
2
Implement safety procedures where needed
Where appropriate, establish procedures for how the chatbot should respond when an interaction moves beyond its intended research role.
3
Test before deployment
Include unusual inputs, edge cases, and attempts to bypass the chatbot’s instructions when testing whether it maintains its intended role and safety boundaries.
cost & usage
1
Consider API costs
API-based implementations may involve usage-based fees. Researchers should consider expected interaction volume, model pricing, and available budget when planning deployment.
2
Monitor API usage
Where relevant, use available provider controls or implementation-level constraints to reduce unintended or excessive API usage.