2024-02-06 14:50:54 Publish script update

app.py

@@ -29,7 +29,7 @@ To do that we added following text to the query:
 
 When  measuring execution time we used `time.time()` result saved to variable  before making the call to API and compared it to `time.time()` result  after receiving the results. We used litellm python library for all of  the models which naturally adds some overhead compared to pure curl  calls.
 
-In order to count tokens we split the output string by  whitespace \w regex character. For data which was impossible to obtain through the API, such as model sizes we only used official sources such  as developers' release blogs and their documentation.
+In order to count tokens we split the output string by  whitespace `\w` regex character. For data which was impossible to obtain through the API, such as model sizes we only used official sources such  as developers' release blogs and their documentation.
 
 When it comes to pricing most providers charge per token count, while HuggingFace Endpoints allow the user to choose machine type and host the model repository on it. The user is then charged by the running time of the machine. In this project we attempted to use HF Endpoints as much as possible due to their popularity and transparency of how the model is executed.
 """
@@ -163,13 +163,21 @@ with gr.Blocks() as demo:
         with gr.Row():
             collapse_languages_button.render()
             collapse_output_method_button.render()
-        summary_ui = gr.DataFrame(dataframe_style(summary_df), label="Statistics")
+        summary_ui = gr.DataFrame(dataframe_style(summary_df), label="Output characteristics")
+        gr.Markdown("""\
+This table compares output characteristics of different models which include execution time, output size and chunking of the output. Some providers and models don't support output chunking, in this case chunk related fields are left empty.
+
+Execution time refers to averaged time needed to execute one query.
+
+To count words we split the output string by whitespace `\w` regex character.
+
+Chunk sizes are measured in the characters count.""")
 
     with gr.Tab("Preformance by time of the day"):
         time_of_day_comparison_ui = gr.DataFrame(dataframe_style(time_of_day_comparison_df), label="Time of day")
         time_of_day_plot_ui = gr.Plot(time_of_day_plot, label="Time of the day plot", scale=1)
         time_periods_explanation_ui = gr.DataFrame(dataframe_style(time_periods_explanation_df), label="Times of day ranges")
-        gr.Markdown("""
+        gr.Markdown("""\
 These measurements were made by testing the models using the same dataset as in the other comparisons every hour for 24 hours.
                     
 Execution time refers to averaged time needed to execute one query.
@@ -189,8 +197,7 @@ Hugging Face Inference Endpoints are charged by hour so to compare different pro
 for models hosted this way we calculated "Cost Per Token" column using data collected during the experiment.
 
 Note that pause and resume time cost was not included in the "Cost Per Token" column calculation.
-            """
-        )
+            """)
     filter_button.click(
         fn=filter_dataframes,
         inputs=filter_textbox,