Linking Cloudflare R2 as a source

Let AI connect your sources for you

Skip the manual setup — run this in your project and the wizard auto-detects your databases and APIs and connects them to PostHog.

Learn more

The Cloudflare R2 source imports CSV, JSON Lines, and Parquet files, and Delta Lake tables, from an R2 bucket into PostHog. PostHog keeps the tables up to date on the schedule you choose, so you can query your files alongside your product data.

Tables you linked before

You can no longer create self-managed tables, which PostHog read directly from your bucket on each query. PostHog moved each existing self-managed table into a source for its storage provider. The table keeps its name, so your queries continue to work.

A moved table has syncing turned off. Until you turn it on, the table continues to read the files in your bucket directly, at no charge. To import the data into PostHog, open the source, find the table, and turn on syncing. PostHog then imports the data on the schedule you choose. Synced rows count toward your data warehouse rows synced.

Prerequisites

You need:

An R2 bucket that contains your files.
An R2 API token with read access to the bucket.

Create an API token

In the Cloudflare dashboard, go to the R2 overview. Under Account details, click Manage R2 API Tokens.
Click Create API token and enter a name for the token.
Select Object Read only as the permission, apply the token to your bucket, and click Create API Token.
Copy the Access Key ID, the Secret Access Key, and the jurisdiction-specific endpoint URL from the credentials for S3 clients. Cloudflare shows these values only once.

Create token

Adding a data source

In PostHog, go to the Sources tab of the data pipeline section.
Click + New source and click Link next to this source.
Enter your credentials (see Configuration below) and click Next.
Select the tables you want to sync, choose a sync method and frequency, then click Import.

Once the syncs are complete, you can start querying this data in PostHog.

When you link R2, enter:

Files URL: The endpoint URL, followed by your bucket and the path to your files, such as https://<account-id>.r2.cloudflarestorage.com/your-bucket/orders/**/*.parquet.
Access key ID and Secret access key: The credentials of the API token that you created.

Files and tables

Files URL

The Files URL tells PostHog which files to import. It starts with the bucket (or container) and can point to a folder, a single file, or a pattern:

Use * to match any part of a file or folder name, such as orders/*.parquet.
Use ** to match across folders, such as orders/**/*.parquet.
A URL without a wildcard imports the folder or file it points to.

The bucket name must be exact. Put wildcards only in the path after the bucket.

Supported file formats

Format	File extensions
CSV, with or without a header row	`.csv`, `.tsv`
JSON Lines (one JSON object per line)	`.jsonl`, `.ndjson`, `.json`
Parquet	`.parquet`
Delta Lake tables	A folder that contains a `_delta_log/` folder

PostHog also reads gzip-compressed files, such as .csv.gz.

By default, PostHog detects the format from the file extension. If your files have no extension or an unusual one, set File format to the format of your files. For CSV files, you can also tell PostHog if double quotes enclose the values. If they don't, PostHog reads quote characters as plain text.

How files become tables

PostHog finds the files that match the URL and creates:

One table for each folder.
One table for each file directly in the folder that the URL points to.
One table for each Delta Lake table.

If all the files share the same columns, such as a daily export, turn on Combine every file into one table? and enter a table name. PostHog then imports all the files into that one table. Delta Lake tables still get one table each.

After you connect, select the tables to import, then choose a sync method and a sync frequency for each table.

Limits

A source can have up to 1,000 tables.
A table can have up to 50,000 files. If a table has more files, the sync fails. Use a more specific URL, or split the files into more folders.

Sync modes

Each table supports three sync methods:

Sync method	What it reads	What happens to existing rows
Full refresh	All the files, on each sync	The table always matches the files in the bucket.
Incremental	Only new and changed files	When you delete a file, PostHog removes its rows from the table. For a Delta Lake table, PostHog also applies deleted and updated rows.
Append only	Only new files that are at least one minute old	PostHog keeps every row that it imported before.

Use Incremental for most tables. Use Append only for folders where files are only added, such as event logs. See sync methods for more about each method.

Rows that PostHog imports count toward your data warehouse rows synced.

Configuration

Option	Description
`Files URL` Type: text Required: True	Use * to match any part of a file or folder name, and ** to match across folders. A URL without a wildcard imports the folder or file it points to.
`Access key ID` Type: text Required: True	Create an R2 API token with read access to the bucket, then copy its S3 credentials.
`Secret access key` Type: password Required: True
`File format` Type: select Required: True
`Combine every file into one table?` Type: switch-group Required: False	Turn this on when the folder holds files that share the same columns, such as a daily export. Delta tables still get a table each.

Supported tables

The tables available from this source are discovered from your account when you connect it, so the exact list depends on your data. Once connected, you can pick which tables to sync from the sources tab.

Troubleshooting

The storage settings are invalid: Make sure that the files URL is in the format that the example shows, and that the bucket name doesn't contain a wildcard.
The storage provider denied access to the bucket: Make sure that the credentials are correct and can list and read the files in the bucket.
The bucket doesn't exist: Make sure that the bucket name in the URL is correct.
No importable files were found: Make sure that the URL matches at least one file in a supported format. If your files have no extension, set File format to the format of your files.
The table has more files than PostHog can import: Use a more specific URL, or split the files into more folders.
Couldn't parse the file: Make sure that File format matches your files. For CSV files, try to change the double quotes setting. For JSON files, make sure each line holds one JSON object, not a single JSON array.
The Delta table uses features PostHog can't read: Some Delta Lake features aren't supported. Export the table to Parquet files and import those files instead.

If your sync is failing or data looks wrong, see the Data warehouse troubleshooting guide. If that doesn't help, contact support – we're happy to help.