Translating medical terminology can be a complex and time-consuming task in which accuracy is vital. ChatGPT offers several ways for users to influence the model’s generated output - features not offered by other machine translation methods currently being used within the medical domain. This thesis explores how ChatGPT performs as a tool for the task of translating a dictionary of medical events. Also investigated is the extent to which user-adjusted model hyperparameters and structured prompting techniques affect the accuracy of medical term translations using ChatGPT.

In the experiments conducted, a Danish language medical terminologies dic- tionary is translated into Swedish using different approaches, with Google Trans- late being used as a baseline for reference, performing the same translation task as ChatGPT.

Results indicate that the adjustable parameters temperature and top_p do influence ChatGPT’s generated output. By instructing the model to take on a persona or including examples in the prompt (few-shot) the translation accuracy scores enhance significantly. The results especially favor Chat GPT’s in-context learning abilities, proving it can outperform Google Translate when the right reference examples are provided. However, none of the experiments generated perfect results, which is a crucial factor in a medical context, where translation accuracy usually is of high importance.