Python agent and Heroku

Heroku is a Platform as a Service (PaaS) solution for hosting web applications in various agent languages, including Python. With New Relic, you can extend Heroku with metrics from New Relic APM, Browser, and Insights.

This document describes special considerations for using Heroku as a hosting service with New Relic's Python agent.

Install the New Relic agent add-on

After deploying your Python app on Heroku, install the New Relic agent:

Via the Heroku website

To install the New Relic add-on through the Heroku website, you must be logged in to Heroku:

  1. From Heroku's Add-on page for New Relic, select a subscription plan.
  2. From Select an app, select your New Relic app.
  3. Give your application a descriptive name with this Heroku toolbelt command:
    heroku config:set NEW_RELIC_APP_NAME='Your Application Name'
  4. Restart your dyno.
  5. Generate some traffic to your app.
Via Heroku toolbelt

To install the New Relic add-on with Heroku the toolbelt:

  1. Run the following command, where $planlevel is the appropriate subscription plan:
    heroku addons:create newrelic:$planlevel
  2. Give your application a descriptive name with this toolbelt command:
    heroku config:set NEW_RELIC_APP_NAME='Your Application Name'
  3. Restart your dyno.
  4. Generate some traffic to your app.

Installing the add-on automatically creates a private New Relic account and configures access for Heroku hosts. New Relic will begin monitoring application performance, end user experience, and host performance collected after the add-on is installed. Within a few minutes, data should start appearing in your APM Overview page.

Upgrade from an existing New Relic installation

If New Relic is already installed, reinstall the add-on using the Heroku toolbelt command.

heroku config:set NEW_RELIC_APP_NAME='Your Application Name'

Install the Python agent

To install a third party Python package such as New Relic on Heroku, use pip. Heroku automatically looks for a requirements.txt file in the root directory of your project. It installs anything listed in that file when you push your project to Heroku.

  1. Create or edit the requirements.txt file, adding the line:

  2. Django users: Modify the web entry of your Procfile, prefixing the value with newrelic-admin run-program. For example:

    web: newrelic-admin run-program gunicorn mysite.wsgi
  3. Push your project up to Heroku.

This will install the New Relic Python agent package with the latest version listed on the Python Package Index (PyPi).

Update the Python agent

Heroku caches packages and does not detect when a newer version of New Relic's Python agent is available. To force an upgrade:

  1. Edit the requirements.txt file by including the specific Python agent version (n.n.n.n) with the package name:

  2. Push your project up to Heroku.

Verify the New Relic add-on

To verify that the New Relic add-on has been enabled, run:

heroku run env | grep NEW_RELIC

This generates a list of New Relic-specific environment variables in Heroku. The Python agent uses these to determine which New Relic account and application data to use for reporting data.

At a minimum, you should see:

NEW_RELIC_APP_NAME=Your app name

The license key is unique to your New Relic account.

Troubleshoot your installation

Within a few minutes of installing and configuring New Relic, data should start appearing in your app's New Relic APM Overview page. If no data appears, test that environment variables are being detected properly by running:

heroku run newrelic-admin validate-config - stdout

This will create a connection back to New Relic and report test transaction data under the application Python Agent Test in your New Relic account. Capture the output from running the test, and use the data to troubleshoot the issue. If you need further assistance, follow the Python agent troubleshooting procedures.

Initialize the Python agent

To initialize the Python agent:

  1. From the root of your project, find the Procfile.
  2. Modify the web entry in your Procfile to define what to do to start up your Python web application.
  3. Refer to the following examples to insert newrelic-admin run-program at the start of the command.
  4. Run your Python web application under the control of the New Relic Python agent's admin script.
Hosting mechanism Example web entry
Flask with the built-in development host
web: newrelic-admin run-program python
Flask with gunicorn
web: newrelic-admin run-program gunicorn -b "$PORT" -w 3 hello:app
Django with gunicorn listed in INSTALLED_APPS
web: newrelic-admin run-program python hellodjango/ run_gunicorn -b "$PORT" -w 3

Avoid using the built-in development hosts of any web framework prior to Python version 2.7.4 or prior to Django 1.4. Instead, use gunicorn or uWSGI.

The WSGI host using the wsgiref module was not fully WSGI compliant for development hosts prior to Python version 2.7.4. This prevented the New Relic Python agent from being able to report correct data.

WSGI application wrapping

New Relic provides automatic wrapping of the the WSGI application point for these web frameworks:

  • Bottle
  • Django
  • Flask

If you are using any of these Python web frameworks, no additional steps are required.

For others, you must modify the Python code file with your WSGI application entry point to wrap the WSGI application object with a New Relic WSGI application wrapper. This will initiate the timing for the web requests received by your application.

If the entry point is this... Do this...
Entry point is a function

Wrap it in a decorator:

  import newrelic.agent

  def application(environ, start_response):
Entry point is a function or object imported from a different module

Wrap it in pre decorator style:

  import myapp
  application = myapp.WSGIHandler()
  application = newrelic.agent.WSGIApplicationWrapper(application)

Track Celery tasks

To record execution time for Celery tasks as background tasks against your web application, wrap the startup of the Celery host with the newrelic-admin command.

Prefix the existing startup command defined by the worker entry in your Procfile:

worker: newrelic-admin run-program python hellodjango/ celeryd -E -B --loglevel=INFO

Debug the Python agent

To begin debugging, collect the log output from the Python agent. Heroku sends Python agent output to standard output and captures it in the web server log.

To get access to the web server log for Heroku, run:

heroku logs

By default the Python agent will log at info level. If New Relic Support requests an alternate level of logging, you must manually add a config variable. For example, to set logging output to debug, run:

heroku config:add NEW_RELIC_LOG_LEVEL=debug

Your application automatically restarts when you change the log level.

The debug log level produces large quantities of output. Be sure to remove this setting as soon as it is no longer required, by running:

heroku config:remove NEW_RELIC_LOG_LEVEL

Edit the agent configuration file

When using Heroku's add-on with New Relic, this automatically sets key environment variables for the Python agent. You can also customize additional settings with the agent configuration file, or use server-side configuration.

Do not add core settings such as the license key, application name, etc. to the configuration file. Heroku automatically adds these settings.

To customize other settings, use the Python agent configuration file with Heroku:

  1. Add the newrelic.ini agent configuration file to the root directory of your project repository that you are pushing up to Heroku: In the [newrelic] section, include the specific configuration setting; for example:

    transaction_tracer.function_trace = mydbm:connect
  2. Commit the configuration file to your repository, and push the change up to Heroku.
  3. Use the heroku config:add command to set the NEW_RELIC_CONFIG_FILE environment variable for your deployed application:

    heroku config:add NEW_RELIC_CONFIG_FILE=newrelic.ini

If you are using the newrelic-admin wrapper program to launch your WSGI host, the settings for your license key, application name, etc., will be picked up from the environment variables set by Heroku. Any additional settings you set in the agent configuration file will also be applied. Then, when the agent registers with New Relic, any server-side configuration will also be merged to create the final configuration the agent will use.

For more help

Additional documentation resources include:

Recommendations for learning more: