text: Skip to main contentğŸ¦œï¸�ğŸ”— LangChainDocsUse casesIntegrationsAPILangSmithJS/TS DocsCTRLKIntegrationsCallbacksChat modelsDocument loadersDocument transformersLLMsMemoryRetrieversText embedding modelsAgent toolkitsAmadeus ToolkitAzure Cognitive Services ToolkitCSV AgentDocument ComparisonGitHubGmail ToolkitJiraJSON AgentMultion ToolkitOffice365 ToolkitOpenAPI agentsNatural Language APIsPandas Dataframe AgentPlayWright Browser ToolkitPowerBI Dataset AgentPython AgentSpark Dataframe AgentSpark SQL AgentSQL Database AgentVectorstore AgentXorbits AgentToolsVector storesGrouped by providerIntegrationsAgent toolkitsSpark Dataframe AgentOn this pageSpark Dataframe AgentThis notebook shows how to use agents to interact with a Spark dataframe and Spark Connect. It is mostly optimized for question answering.NOTE: this agent calls the Python agent under the hood, which executes LLM generated Python code - this can be bad if the LLM generated Python code is harmful. Use cautiously.import osos.environ["OPENAI_API_KEY"] = "...input your openai api key here..."from langchain.llms import OpenAIfrom pyspark.sql import SparkSessionfrom langchain.agents import create_spark_dataframe_agentspark = SparkSession.builder.getOrCreate()csv_file_path = "titanic.csv"df = spark.read.csv(csv_file_path, header=True, inferSchema=True)df.show()    23/05/15 20:33:10 WARN Utils: Your hostname, Mikes-Mac-mini.local resolves to a loopback address: 127.0.0.1; using 192.168.68.115 instead (on interface en1)    23/05/15 20:33:10 WARN Utils: Set SPARK_LOCAL_IP if you need to bind to another address    Setting default log level to "WARN".    To adjust logging level use