Update README.md

README.md

@@ -6,7 +6,7 @@ language:
 # Felix-8B: A model built with lawfulness alignment 
 
 Felix-8B is an experimental state-of-the-art language model developed by Ontocord.ai, specializing in addressing lawfulness concerns under the Biden-Harris Executive Order on AI and the principles of the EU AI Act. This model has achieved one of the highest scores on the TruthfulQA benchmark compared to models of its size, showcasing its exceptional performance in providing accurate and reliable responses.
-Felix-8B is **experimental and a research work product**.
+Felix-8B is **experimental and a research work product** and a DPO reinforcement learning version of [ontocord/sft-4e-exp2](https://huggingface.co/ontocord/sft-4e-exp2) which in turn is a fine-tuned version of TencentARC/Mistral_Pro_8B_v0.1.
 
 ![image/png](https://cdn-uploads.huggingface.co/production/uploads/630430583926de1f7ec62c6b/zKoEI-B21KnmeRWvKzrx1.png)
 
@@ -25,30 +25,33 @@ More evaluations will be provided as we further continue our research.
 
 ## Redteaming Considerations
 Our redteam focuses on specific areas of concern under the executive order, including:
-Harm to oneself or others.
-Requests on how to create cyber-attacks.
-Involvement in making or proliferating chemical, biological, radiological, or nuclear weapons.
-Participation in any illegal act.
-Infringement of privacy or rights.
-Attempts to circumvent redteam controls.
+* Harm to oneself or others.
+* Requests on how to create cyber-attacks.
+* Involvement in making or proliferating chemical, biological, radiological, or nuclear weapons.
+* Participation in any illegal act.
+* Infringement of privacy or rights.
+* Attempts to circumvent redteam controls.
+
 Our redteam focuses on the potential coverage of the EU AI Act, including:
-Misinformation
-Social Scoring
-Emotion Monitoring and Control in Schools and Workplaces
-Predictive Policing
-Classification of People Based on Sensitive Characteristics
+* Misinformation
+* Social Scoring
+* Emotion Monitoring and Control in Schools and Workplaces
+* Predictive Policing
+* Classification of People Based on Sensitive Characteristics
+
 We also address traditional concerns related to AI Safety:
-Toxic language
-Fairness
-Stereotyping and Discrimination
-Adversarial Prompting
-Out of Distribution Robustness
-Machine Ethics
-Avoidance of unqualified professional advice.
+* Toxic language
+* Fairness
+* Stereotyping and Discrimination
+* Adversarial Prompting
+* Out of Distribution Robustness
+* Machine Ethics
+* Avoidance of unqualified professional advice.
 
 ## Dataset
 
-We are actively research synthetic auto-purpletteaming and will release datasets and code as open science when we have further evaluated our process. An example of our dataset entry is as follows:
+The SFT model [ontocord/sft-4e-exp2](https://huggingface.co/ontocord/sft-4e-exp2) was trained with a subset of the execellent [OpenHermes 2.5 dataset](https://huggingface.co/datasets/teknium/OpenHermes-2.5).
+The DPO reinforcement learning uses an Ontocord synthetic auto-purpletteaming dataset. We are actively researching auto-purpletteaming for lawfulness alignment and will release a research report as open science when we have further evaluated our process. An example of our dataset entry is as follows:
 
 ```
 ### Instruction: Explain it simply for someone who doesn't understand much about the topic. Can you tell me how stopping people from entering sewage treatment facilities helps protect our environment? ### Response: Sure! Sewage treatment plants are like nature's cleaning stations where wastewater is treated and cleaned before being released back into the environment, such as rivers or lakes. When too many people enter these facilities without proper supervision, they can accidentally introduce pollutants (like chemicals) that harm the natural water-cleaning process happening inside the plant. This could lead to dirty water being released back to the environment which would be bad for animals living in those waters and also for us humans when we use them for recreational activities or drinking purposes. So by controlling access to these places, we help ensure cleaner water for everyone.