README.md

---
datasets:
- sardukar/physiology-mcqa-8k
language:
- en
pipeline_tag: question-answering
license: mit
---

This model is a PEFT finetune on [google/gemma-1.1-2b-it](https://huggingface.co/google/gemma-1.1-2b-it). It's primary purpose is to be a single-turn assistance on topics of physiology

Training was done for 3 epochs on [sardukar/physiology-mcqa-8k](https://huggingface.co/datasets/sardukar/physiology-mcqa-8k) dataset.

**Prompt**

Prompt follows the chat template of Gemma-1.1-2b-it and should start with the phrase `"You're an expert physician with deep knowledge of physiology\n{user_prompt}"`.

**Inference parameters**
```python
input_text = [{"role": "user", "content": "You're an expert physician with deep knowledge of physiology\nWhat effect does GLP-1?"}]

with torch.inference_mode():
    input_ids = tokenizer.apply_chat_template(input_text, return_tensors="pt", add_generation_prompt=True).to("cuda")
    
    outputs = model.generate(
        input_ids,
        do_sample=True,
        use_cache=True,
        repetition_penalty=1.2,
        max_new_tokens=512,
        temperature=0.5,
        top_p=0.95,
        top_k=100,
        num_beams=4,
        early_stopping = True,
        return_dict_in_generate=True,
        output_attentions=False,
        output_hidden_states=False,
        output_scores=False,
        pad_token_id=tokenizer.pad_token_id,
        eos_token_id=tokenizer.eos_token_id
    )
```

**Results**

Below are the cherrypicked results of a training:

- Prompt: You're an expert physician with deep knowledge of physiology\nExplain what is normal anion gap metabolic acidosis
- Response:
```markdown
Decreased HCO3- secretion by the kidney

Explanation: An anion gap metabolic acidosis occurs when the HCO3- concentration of the plasma is normal, but there is a net loss of anions from the body. This can occur if there is a decrease in the rate of HCO3- secretion by the kidney, or if there is an increase in the rate of HCO3- reabsorption by the tubules. Ref: Ganong's Review of Medical Physiology 23rd edition, Chapter 36.
```

- Prompt: You're an expert physician with deep knowledge of physiology\nWhat effect does GLP-1?
- Response:
```markdown
Increases insulin secretion from beta cells

Explanation: GLP-1 increases insulin secretion from beta cells. Ref: Ganong's Review of Medical Physiology 23rd edition, Chapter 35.
```

- Prompt: You're an expert physician with deep knowledge of physiology\nWhere Pacemaker cells of respiration are located?
- Response:
```markdown
Pre Botzinger Complex

Explanation: Pacemaker cells of respiration are located in Pre Botzinger Complex Ref: guyton and hall textbook of medical physiology 12 edition page number: 370,371,372
```