Pipelines - PGFS

Pipelines has a "Postgres File System" (PGFS) that allows you access external storage. In particular, it allows you to store and retrieve files from either a public or private S3-compatible bucket or a local file system.

PGFS is a key component of Pipelines, allowing you to work with data which is not stored in the database, either because it is too large or because it is not structured in a way that is suitable for a relational database.

PGFS is implemented as a foreign data wrapper (FDW) and is installed as an extension. The extension provides a set of functions to interact with the file system from within the database.

Installing the PGFS extension

See Installing for instructions on how to install the PGFS extension as part of the Pipelines installation.

Using PGFS

Depending on where the data is stored, you will need to create a server object to connect to the storage location. The server object is created using the CREATE SERVER command.

• S3-compatible storage
• Local file system