here we go :rocket:

README.md

@@ -1,8 +1,8 @@
 ---
-title: Biodiversity Justice
+title: Geo Llm R
 emoji: 📚
 colorFrom: blue
-colorTo: green
+colorTo: yellow
 sdk: docker
 pinned: false
 license: bsd-2-clause
@@ -12,3 +12,32 @@ license: bsd-2-clause
 
 :hugs: Shiny App on Huggingface: <https://huggingface.co/spaces/boettiger-lab/geo-llm-r>
 
+Work in progress.  This is a proof-of-principle for an LLM-driven interface to dynamic mapping. Key technologies include duckdb, geoparquet, pmtiles, maplibre, open LLMs (via VLLM + LiteLLM).  R interface through ellmer (LLMs), mapgl (maplibre), shiny, and duckdb.
+
+# Setup
+
+## GitHub with HuggingFace Deploy
+
+All edits should be pushed to GitHub. Edits to `main` branch are automatically deployed to HuggingFace via GitHub Actions.
+When using this scaffold, you will first have to set up your auto-deploy system: 
+
+- [Create a new HuggingFace Space](https://huggingface.co/new-space) (any template is fine, will be overwritten).
+- [Create a HuggingFace Token](https://huggingface.co/settings/tokens/new?tokenType=write) with write permissions if you do not have one.  
+- In the GitHub Settings of your repository, add the token as a "New Repository Secret" under the `Secrets and Variables` -> `Actions` section of settings (`https://github.com/{USER}/{REPO}/settings/secrets/actions`).  
+- Edit the `.github/workflows/deploy.yml` file to specify your HuggingFace user name and HF repo to publish to. 
+
+## Language Model setup
+
+This example is designed to be able to leverage open source or open weights models.  You will need to adjust the API URL and API key accordingly. This could be a local model with `vllm` or `ollama`, and of course commercial models should work too. The demo app currently runs on an VLLM+LiteLLM backed model, currently a Llama3 variant, hosted on the National Research Platform.
+
+The LLM plays only a simple role in generating SQL queries from background information on the data including the table schema, see the system prompt for details. Most open models I have experimented with do not support the [tool use](https://ellmer.tidyverse.org/articles/tool-calling.html) or [structured data](https://ellmer.tidyverse.org/articles/structured-data.html) interfaces very well compared to commercial models.  An important trick in working with open models used here is merely requesting the reply be structured as JSON.  Open models are quite decent at this, and at SQL construction, given necessary context about the data. The map and chart elements merely react the resulting data frames, and the entire analysis is thus transparent and reproducible as it would be if the user had composed their request in SQL instead of plain English. 
+
+## Software Dependencies
+
+The Dockerfile includes all dependencies required for the HuggingFace deployment, and can be used as a template or directly to serve RStudio server.
+
+## Data pre-processing
+
+Pre-processing the data into cloud-native formats and hosting data on a high bandwidth, highly avalialbe server is essential for efficient and scalable renending.  Pre-computing expensive operations such as zonal statistics across all features is also necessary. These steps are described in [preprocess.md](preprocess.md) and corresponding scripts.
+
+

app.R

@@ -1,20 +1,23 @@
 library(shiny)
 library(bslib)
 library(htmltools)
-library(markdown)
+#library(markdown)
 library(fontawesome)
 library(bsicons)
 library(gt)
 library(glue)
 library(ggplot2)
-
-library(mapgl)
+library(readr)
 library(dplyr)
+library(mapgl)
 library(duckdbfs)
-
 duckdbfs::load_spatial()
 
-css <- HTML("<link rel='stylesheet' type='text/css' href='https://demos.creative-tim.com/material-dashboard/assets/css/material-dashboard.min.css?v=3.2.0'>")
+css <-
+  HTML(paste0("<link rel='stylesheet' type='text/css' ",
+              "href='https://demos.creative-tim.com/",
+              "material-dashboard/assets/css/",
+              "material-dashboard.min.css?v=3.2.0'>"))
 
 
 # Define the UI
@@ -23,23 +26,28 @@ ui <- page_sidebar(
   tags$head(css),
   titlePanel("Demo App"),
 
-"This is a proof-of-principle for a simple chat-driven interface to dynamically explore geospatial data. 
- ",
-
+  "
+  This is a proof-of-principle for a simple chat-driven interface
+  to dynamically explore geospatial data.
+  ",
 
   card(
     layout_columns(
-    textInput("chat",
-      label = NULL,
-      "Which counties in California have the highest average social vulnerability?",
-      width = "100%"),
-    div(
-    actionButton("user_msg", "", icon = icon("paper-plane"),
-                 class = "btn-primary btn-sm align-bottom"),
-    class = "align-text-bottom"),
-    col_widths = c(11, 1)),
-    fill = FALSE
+      textInput("chat",
+        label = NULL,
+        "Which counties in California have the highest average social vulnerability?",
+        width = "100%"),
+      div(
+      actionButton("user_msg", "", icon = icon("paper-plane"),
+                   class = "btn-primary btn-sm align-bottom"),
+      class = "align-text-bottom"),
+      col_widths = c(11, 1)),
+      fill = FALSE
   ),
+
+  textOutput("agent"),
+
+
   layout_columns(
     card(maplibreOutput("map")),
     card(includeMarkdown("## Plot"),
@@ -51,12 +59,10 @@ ui <- page_sidebar(
     max_height = "700px"
   ),
 
-
   gt_output("table"),
 
   card(fill = TRUE,
     card_header(fa("robot")),
- 
     accordion(
       open = FALSE,
       accordion_panel(
@@ -70,34 +76,13 @@ ui <- page_sidebar(
         textOutput("explanation"),
       )
     ),
-
     card(
       card_header("Errata"),
-      markdown(
-"
-#### Credits
-
-Developed by Carl Boettiger, UC Berkeley, 2025.  BSD License.
-
-Data from the US Census and CDC's [Social Vulnerability Index](https://www.atsdr.cdc.gov/place-health/php/svi/index.html)
-
-#### Technical details
-
-The app is written entirely in R using shiny. The app will translate natural language queries in SQL code using
-a small open-weights language model. The SQL code is executed using the duckdb backend against cloud-native
-geoparquet snapshot of the Social Vulnerability Index hosted on Source Cooperative. Summary chart data are also
-computed in duckdb by streaming, providing responsive updates while needing minimal RAM or disk storage despite
-the large size of the data sources. 
-
-The map is rendered and updated using MapLibre with PMTiles, which provides responsive rendering for large feature sets.
-The PMTiles layer is also hosted on Source cooperative where it can be streamed efficiently.
-")
+      shiny::markdown(readr::read_file("footer.md")),
     )
-
   ),
 
   sidebar = sidebar(
-
     input_switch("redlines", "Redlined Areas", value = FALSE),
     input_switch("svi", "Social Vulnerability", value = TRUE),
     input_switch("richness", "Biodiversity Richness", value = FALSE),
@@ -112,41 +97,15 @@ The PMTiles layer is also hosted on Source cooperative where it can be streamed
   theme = bs_theme(version = "5")
 )
 
-
-
-
 repo <- "https://data.source.coop/cboettig/social-vulnerability"
-pmtiles <- glue("{repo}/svi2020_us_tract.pmtiles")
-parquet <- glue("{repo}/svi2020_us_tract.parquet")
-svi <- open_dataset(parquet, tblname = "svi") |>
-  filter(RPL_THEMES > 0)
-
+pmtiles <- glue("{repo}/2022/SVI2022_US_county.pmtiles")
 
+duckdb_s3_config(s3_endpoint = "minio.carlboettiger.info")
 con <- duckdbfs::cached_connection()
-schema <- DBI::dbGetQuery(con, "PRAGMA table_info(svi)")
-
-system_prompt = glue::glue('
-You are a helpful agent who always replies strictly in JSON-formatted text.
-Your task is to translate the users question into a SQL query that will be run
-against the "svi" table in a duckdb database. The duckdb database has a
-spatial extension which understands PostGIS operations as well. 
-Include semantically meaningful columns like COUNTY and STATE name.
-
-In the data, each row represents an individual census tract. If asked for 
-county or state level statistics, be sure to aggregate across all the tracts
-in that county or state. 
-
-The table schema is <schema>
-
-The column called "RPL_THEMES" corresponds to the overall "Social vulnerability index" number. 
-
-Format your answer as follows:
-
-{
-"query": "your raw SQL response goes here",
-"explanation": "your explanation of the query"
-}
-', .open = "<", .close = ">")
+svi <-  open_dataset("s3://public-gbif/svi", tblname = "svi") |> filter(RPL_THEMES > 0)
+schema <- read_file("schema.yml")
+system_prompt <- glue::glue(readr::read_file("system-prompt.md"),
+                            .open = "<", .close = ">")
 
 chat <- ellmer::chat_vllm(
   base_url = "https://llm.nrp-nautilus.io/",
@@ -168,52 +127,64 @@ filter_column <- function(full_data, filtered_data, id_col = "FIPS") {
   list("in", list("get", id_col), list("literal", values))
 }
 
-chart1_data <- svi |>
-  group_by(COUNTY) |>
-  summarise(mean_svi = mean(RPL_THEMES)) |>
-  collect()
-
-chart1 <- chart1_data |>
-  ggplot(aes(mean_svi)) + geom_density(fill="darkred") +
-  ggtitle("County-level vulnerability nation-wide")
 
 
 # Define the server
 server <- function(input, output, session) {
+
+  chart1_data <- svi |>
+    group_by(COUNTY) |>
+    summarise(mean_svi = mean(RPL_THEMES)) |>
+    collect()
+
+  chart1 <- chart1_data |>
+    ggplot(aes(mean_svi)) + geom_density(fill="darkred") +
+    ggtitle("County-level vulnerability nation-wide")
+
   data <- reactiveValues(df = tibble())
   output$chart1 <- renderPlot(chart1)
 
   observeEvent(input$user_msg, {
     stream <- chat$chat(input$chat)
 
-    # optional, remember previous discussion
-    #chat_append("chat", stream)
+
 
     # Parse response
     response <- jsonlite::fromJSON(stream)
-    output$sql_code <- renderText(stringr::str_wrap(response$query, width = 60))
-    output$explanation <- renderText(response$explanation)
 
-    # Actually execute the SQL query generated:
-    df <- DBI::dbGetQuery(con, response$query)
+    if ("query" %in% names(response)) {
+      output$sql_code <- renderText(stringr::str_wrap(response$query, width = 60))
+      output$explanation <- renderText(response$explanation)
+
+      # Actually execute the SQL query generated:
+      df <- DBI::dbGetQuery(con, response$query)
 
-    # don't display shape column in render
-    df <- df |> select(-any_of("Shape"))
-    output$table <- render_gt(df, height = 300)
+      # don't display shape column in render
+      df <- df |> select(-any_of("Shape"))
+      output$table <- render_gt(df, height = 300)
 
 
-    y_axis <- colnames(df)[!colnames(df) %in% colnames(svi)] 
-    chart2 <- df |> 
-      rename(social_vulnerability = y_axis) |>
-      ggplot(aes(social_vulnerability)) +
-      geom_density(fill = "darkred")  +
-      xlim(c(0, 1)) +
-      ggtitle("Vulnerability of selected areas")
+      y_axis <- colnames(df)[!colnames(df) %in% colnames(svi)]
+      chart2 <- df |>
+        rename(social_vulnerability = y_axis) |>
+        ggplot(aes(social_vulnerability)) +
+        geom_density(fill = "darkred")  +
+        xlim(c(0, 1)) +
+        ggtitle("Vulnerability of selected areas")
 
-    output$chart2 <- renderPlot(chart2)
+      output$chart2 <- renderPlot(chart2)
 
-    # We need to somehow trigger this df to update the map.
-    data$df <- df
+      # We need to somehow trigger this df to update the map.
+      data$df <- df
+
+    # Note: ellmer will preserve full chat history automatically.
+    # this can confuse the agent and mess up behavior, so we reset:
+    chat$set_turns(NULL)
+
+    } else {
+      output$agent <- renderText(response$agent)
+
+    }
 
   })
 
@@ -259,7 +230,7 @@ server <- function(input, output, session) {
           id = "svi_layer",
           source = list(type = "vector",
                         url = paste0("pmtiles://", pmtiles)),
-          source_layer = "SVI2000_US_tract",
+          source_layer = "svi",
           filter = filter_column(svi, data$df, "FIPS"),
           fill_opacity = 0.5,
           fill_color = interpolate(column = "RPL_THEMES",

footer.md

@@ -0,0 +1,16 @@
+#### Credits
+
+Developed by Carl Boettiger, UC Berkeley, 2025.  BSD License.
+
+Data from the US Census and CDC's [Social Vulnerability Index](https://www.atsdr.cdc.gov/place-health/php/svi/index.html)
+
+#### Technical details
+
+The app is written entirely in R using shiny. The app will translate natural language queries in SQL code using
+a small open-weights language model. The SQL code is executed using the duckdb backend against cloud-native
+geoparquet snapshot of the Social Vulnerability Index hosted on Source Cooperative. Summary chart data are also
+computed in duckdb by streaming, providing responsive updates while needing minimal RAM or disk storage despite
+the large size of the data sources. 
+
+The map is rendered and updated using MapLibre with PMTiles, which provides responsive rendering for large feature sets.
+The PMTiles layer is also hosted on Source cooperative where it can be streamed efficiently.

preprocess.md

@@ -0,0 +1,27 @@
+---
+---
+
+
+# Vector Layers
+
+The heart of this application design is a vector dataset serialized as both (Geo)Parquet and PMTiles.
+The parquet version allows for real-time calculations through rapid SQL queries via duckdb, 
+and the PMTiles version allows the data to be quickly visualized at any zoom through maplibre.
+maplibre can also efficiently filter the PMTiles data given a feature ids returned by duckdb.
+
+`gdal_translates` can generate both PMTiles and geoparquet, though `tippecanoe` provides more
+options for PMTiles generation and can produce nicer tile sets.
+
+The demo uses the CDC Social Vulnerability data because it is built on the hierachical partitioning
+used by the Census (Country->State->County->Tract) hierarchy.  
+
+# Raster Layers
+
+## Generating static tiles
+
+## Zonal statistics calculations 
+
+The application is essentially driven by the vector layer data using SQL. 
+I find it helpful to pre-process 'zonal' calculations, e.g. the mean value of each raster layer
+within each feature in the 'focal' vector data set(s).
+

schema.yml

@@ -0,0 +1,41 @@
+- VARIABLE_NAME: ST
+  DESCRIPTION: State-level FIPS code (two-digit integer)
+- VARIABLE_NAME: STATE
+  DESCRIPTION: State name
+- VARIABLE_NAME: ST_ABBR
+  DESCRIPTION: State abbreviation
+- VARIABLE_NAME: STCNTY
+  DESCRIPTION: County-level FIPS code (5 digit integer)
+- VARIABLE_NAME: COUNTY
+  DESCRIPTION: County name
+- VARIABLE_NAME: FIPS
+  DESCRIPTION: Tract-level geographic identification (full Census Bureau FIPS code)
+- VARIABLE_NAME: LOCATION
+  DESCRIPTION: Text description of tract county state
+- VARIABLE_NAME: AREA_SQMI
+  DESCRIPTION: Tract area in square miles
+- VARIABLE_NAME: RPL_THEMES
+  DESCRIPTION: Overall social vulnerability. Should always be used unless explicit sub-theme is called for.
+- VARIABLE_NAME: RPL_THEME1
+  DESCRIPTION: Subtheme for socio-economic status social vulnerability score
+- VARIABLE_NAME: RPL_THEME2
+  DESCRIPTION: Subtheme for Household characteristics vulnerability score
+- VARIABLE_NAME: RPL_THEME3
+  DESCRIPTION: Subtheme for Racial and Ethnic Minority status based vulnerability score
+- VARIABLE_NAME: RPL_THEME4
+  DESCRIPTION: Subtheme for Housing and transportation-based vulnerability score
+- VARIABLE_NAME: kingdom
+  DESCRIPTION: phylogenetic kingdom
+- VARIABLE_NAME: phylum
+  DESCRIPTION: phylogenetic phylum
+- VARIABLE_NAME: class
+  DESCRIPTION: phylogenetic class
+- VARIABLE_NAME: order
+  DESCRIPTION: phylogenetic order
+- VARIABLE_NAME: family
+  DESCRIPTION: phylogenetic family
+- VARIABLE_NAME: genus
+  DESCRIPTION: phylogenetic genus
+- VARIABLE_NAME: species
+  DESCRIPTION: phylogenetic genus and species name (scientific name)
+

system-prompt.md

@@ -0,0 +1,34 @@
+
+You are a helpful agent who always replies strictly in JSON-formatted text.
+Your task is to translate the user's questions about the data into a SQL query
+that will be run against the "biodiversity_occurrences" table in a duckdb database.
+The duckdb database has a spatial extension which understands PostGIS operations as well.
+
+If your answer involves the construction of a SQL query, you must format your answer as follows:
+
+{
+"query": "your raw SQL response goes here",
+"explanation": "your explanation of the query"
+}
+
+If your answer does not involve a SQL query, please reply with the following format instead:
+
+{
+    "user": "user question goes here",
+    "agent": "your response goes here"
+}
+
+If you are asked to describe the data or for information about the data schema, give only a human-readable response with SQL.
+
+In the data, each row represents an individual occurrence of a species. The occurrences
+are geocoded to US Census counties, with the STATE, COUNTY, and FIPS columns indicating
+the corresponding state name, county name, and FIPS identifier for the specific County.
+The FIPS column is an 5-digit number that uniquely identifies a county in a state.
+Taxonomic classification of the species is given in the corresponding columns, kingdom,
+phylum, class, order, family, genus, and species. 
+
+The data also includes information about various measures of social vulnerability (RPL_THEMES). 
+Pay attention to the DESCRIPTION of each of the columns (VARIABLE_NAME) from the metadata table:
+<schema>
+
+

test.R

@@ -0,0 +1,82 @@
+## Illustrate/test core app functionality without shiny
+
+library(tidyverse)
+library(duckdbfs)
+library(mapgl)
+library(ellmer)
+library(glue)
+
+repo <- "https://data.source.coop/cboettig/social-vulnerability"
+pmtiles <- glue("{repo}/svi2020_us_tract.pmtiles")
+duckdb_s3_config(s3_endpoint = "minio.carlboettiger.info")
+svi <-
+  open_dataset("s3://public-gbif/svi", tblname = "biodiversity_occurrences") |>
+  filter(RPL_THEMES > 0)
+schema <- read_file("schema.yml")
+system_prompt <- glue::glue(readr::read_file("system-prompt.md"),
+                            .open = "<", .close = ">")
+
+
+
+
+# Or optionally test with cirrus
+chat <- ellmer::chat_vllm(
+  base_url = "https://llm.cirrus.carlboettiger.info/v1/",
+  model = "kosbu/Llama-3.3-70B-Instruct-AWQ",
+  api_key = Sys.getenv("CIRRUS_LLM_KEY"),
+  system_prompt = system_prompt,
+  api_args = list(temperature = 0)
+)
+
+# or use the NRP model
+chat <- ellmer::chat_vllm(
+  base_url = "https://llm.nrp-nautilus.io/",
+  model = "llama3",
+  api_key = Sys.getenv("NRP_API_KEY"),
+  system_prompt = system_prompt,
+  api_args = list(temperature = 0)
+)
+
+cols <- colnames(svi)
+rpls <- grep("RPL_THEME.+", cols)
+keep <- cols[c(1:9, rpls, 161:226)]
+biodiversity <- svi |> select(all_of(keep))
+
+# Test a chat-based response
+chat$chat("Which columns describes racial components of social vulnerability?")
+chat$set_turns(NULL)
+## A query-based response
+stream <- chat$chat("Which counties have the most bird observations?")
+stream <- chat$chat("Give me the number bird observations per county vs county social vulnerability")
+response <- jsonlite::fromJSON(stream)
+
+
+stream2 <- chat$chat("Great, now give me the ggplot2 code to plot the data.frame you returned as those counts vs social vulnerability as points.  Be sure to place the R code for your reply by itself in a 'code' element of the JSON")
+
+response <- jsonlite::fromJSON(stream2)
+
+con <- duckdbfs::cached_connection()
+filtered_data <- DBI::dbGetQuery(con, response$query)
+
+filter_column <- function(full_data, filtered_data, id_col) {
+  if (nrow(filtered_data) < 1) return(NULL)
+  values <- full_data |>
+    inner_join(filtered_data, copy = TRUE) |>
+    pull(id_col)
+  # maplibre syntax for the filter of PMTiles  
+  list("in", list("get", id_col), list("literal", values))
+}
+
+maplibre(center = c(-102.9, 41.3), zoom = 3) |>
+    add_fill_layer(
+        id = "svi_layer",
+        source = list(type = "vector", url  = paste0("pmtiles://", pmtiles)),
+        source_layer = "SVI2000_US_tract",
+        filter = filter_column(full_data, filtered_data, "FIPS"),
+        fill_opacity = 0.5,
+        fill_color = interpolate(column = "RPL_THEMES",
+                                values = c(0, 1),
+                                stops = c("#e19292c0", "darkblue"),
+                                na_color = "lightgrey")
+    )
+

utils.R

@@ -0,0 +1,4 @@
+
+library(tidyverse)
+library(duckdbfs)
+