Create README.md

README.md

@@ -0,0 +1,71 @@
+---
+datasets:
+- sardukar/physiology-mcqa-8k
+language:
+- en
+pipeline_tag: question-answering
+license: mit
+---
+
+This model is a PEFT finetune on [google/gemma-1.1-2b-it](https://huggingface.co/google/gemma-1.1-2b-it). It's primary purpose is to be a single-turn assistance on topics of physiology
+
+Training was done for 3 epochs on [sardukar/physiology-mcqa-8k](https://huggingface.co/datasets/sardukar/physiology-mcqa-8k) dataset.
+
+**Prompt**
+
+Prompt follows the chat template of Gemma-1.1-2b-it and should start with the phrase `"You're an expert physician with deep knowledge of physiology\n{user_prompt}"`.
+
+**Inference parameters**
+```python
+input_text = [{"role": "user", "content": "You're an expert physician with deep knowledge of physiology\nWhat effect does GLP-1?"}]
+
+with torch.inference_mode():
+    input_ids = tokenizer.apply_chat_template(input_text, return_tensors="pt", add_generation_prompt=True).to("cuda")
+    
+    outputs = model.generate(
+        input_ids,
+        do_sample=True,
+        use_cache=True,
+        repetition_penalty=1.2,
+        max_new_tokens=512,
+        temperature=0.5,
+        top_p=0.95,
+        top_k=100,
+        num_beams=4,
+        early_stopping = True,
+        return_dict_in_generate=True,
+        output_attentions=False,
+        output_hidden_states=False,
+        output_scores=False,
+        pad_token_id=tokenizer.pad_token_id,
+        eos_token_id=tokenizer.eos_token_id
+    )
+```
+
+**Results**
+
+Below are the cherrypicked results of a training:
+
+- Prompt: You're an expert physician with deep knowledge of physiology\nExplain what is normal anion gap metabolic acidosis
+- Response:
+```markdown
+Decreased HCO3- secretion by the kidney
+
+Explanation: An anion gap metabolic acidosis occurs when the HCO3- concentration of the plasma is normal, but there is a net loss of anions from the body. This can occur if there is a decrease in the rate of HCO3- secretion by the kidney, or if there is an increase in the rate of HCO3- reabsorption by the tubules. Ref: Ganong's Review of Medical Physiology 23rd edition, Chapter 36.
+```
+
+- Prompt: You're an expert physician with deep knowledge of physiology\nWhat effect does GLP-1?
+- Response:
+```markdown
+Increases insulin secretion from beta cells
+
+Explanation: GLP-1 increases insulin secretion from beta cells. Ref: Ganong's Review of Medical Physiology 23rd edition, Chapter 35.
+```
+
+- Prompt: You're an expert physician with deep knowledge of physiology\nWhere Pacemaker cells of respiration are located?
+- Response:
+```markdown
+Pre Botzinger Complex
+
+Explanation: Pacemaker cells of respiration are located in Pre Botzinger Complex Ref: guyton and hall textbook of medical physiology 12 edition page number: 370,371,372
+```