PostgreSQL Database Backup

posted Sep 17, 2010, 2:41 PM by Sachchida Ojha
pg_dump is a utility for backing up a PostgreSQL database. It makes consistent backups even if the database
is being used concurrently. pg_dump does not block other users accessing the database (readers or writers).

Dumps can be output in script or archive file formats. Script dumps are plain-text files containing the SQL
commands required to reconstruct the database to the state it was in at the time it was saved.
To restore from such a script, feed it to psql. Script files can be used to reconstruct the database
even on other machines and other architectures; with some modifications even on other SQL database products.


The alternative archive file formats must be used with pg_restore to rebuild the database.
They allow pg_restore to be selective about what is restored, or even to reorder the items prior to being
restored. The archive file formats are designed to be portable across architectures.

When used with one of the archive file formats and combined with pg_restore, pg_dump provides
a flexible archival and transfer mechanism. pg_dump can be used to backup an entire database,
then pg_restore can be used to examine the archive and/or select which parts of the database
are to be restored. The most flexible output file format is the "custom" format (-Fc).
It allows for selection and reordering of all archived items, and is compressed by default.
The tar format (-Ft) is not compressed and it is not possible to reorder data when loading,
but it is otherwise quite flexible; moreover, it can be manipulated with standard Unix tools such as tar.

While running pg_dump, one should examine the output for any warnings (printed on standard error),

especially in light of the limitations listed below.
Options

The following command-line options control the content and format of the output.

dbname

Specifies the name of the database to be dumped. If this is not specified, the environment
variable PGDATABASE is used. If that is not set, the user name specified for the connection is used.
-a
--data-only

    Dump only the data, not the schema (data definitions).
    This option is only meaningful for the plain-text format. For the archive formats, you may
    specify the option when you call pg_restore.
-c
--clean
    Output commands to clean (drop) database objects prior to (the commands for) creating them.
    This option is only meaningful for the plain-text format. For the archive formats, you
    may specify the option when you call pg_restore.
-C
--create
    Begin the output with a command to create the database itself and reconnect to the created database.
    (With a script of this form, it doesn't matter which database you connect to before running the script.)
    This option is only meaningful for the plain-text format. For the archive formats, you may
    specify the option when you call pg_restore.
-d
--inserts
    Dump data as INSERT commands (rather than COPY). This will make restoration very slow;
    it is mainly useful for making dumps that can be loaded into non-PostgreSQL databases.
     Note that the restore may fail altogether if you have rearranged column order.
    The -D option is safer, though even slower. Also, while this option generates errors for
     invalid data, it allows other INSERTs to continue loading data into the table.
-D
--column-inserts
--attribute-inserts
    Dump data as INSERT commands with explicit column names (INSERT INTO table (column, ...) VALUES ...).
    This will make restoration very slow; it is mainly useful for making dumps that can be loaded
    into non-PostgreSQL databases. Also, while this option generates errors for invalid data,
     it allows other INSERTs to continue loading data into the table.
-E encoding
--encoding=encoding
    Create the dump in the specified character set encoding. By default, the dump is created in the
    database encoding. (Another way to get the same result is to set the PGCLIENTENCODING environment
    variable to the desired dump encoding.)
-f file
--file=file
    Send output to the specified file. If this is omitted, the standard output is used.
-F format
--format=format
    Selects the format of the output. format can be one of the following:
    p
        Output a plain-text SQL script file (default)
    t
        Output a tar archive suitable for input into pg_restore. Using this archive format allows reordering and/or
        exclusion of database objects at the time the database is restored. It is also possible to limit which
        data is reloaded at restore time.
    c
    Output a custom archive suitable for input into pg_restore. This is the most flexible format in that
        it allows reordering of loading data as well as object definitions. This format is also compressed by default.
-i
--ignore-version
    Ignore version mismatch between pg_dump and the database server.
    pg_dump can handle databases from previous releases of PostgreSQL, but very old versions are not
    supported anymore (currently prior to 7.0). Use this option if you need to override the version check
    (and if pg_dump then fails, don't say you weren't warned).
-n schema
--schema=schema
    Dump the contents of schema only. If this option is not specified, all non-system schemas in the target
    database will be dumped.
        Note: In this mode, pg_dump makes no attempt to dump any other database objects that objects
        in the selected schema may depend upon. Therefore, there is no guarantee that the results of
        a single-schema dump can be successfully restored by themselves into a clean database.
-o
--oids
    Dump object identifiers (OIDs) as part of the data for every table. Use this option if your application
    references the OID columns in some way (e.g., in a foreign key constraint). Otherwise, this option
    should not be used.
-O
--no-owner

   Do not output commands to set ownership of objects to match the original database.
    By default, pg_dump issues ALTER OWNER or SET SESSION AUTHORIZATION statements to set
    ownership of created database objects. These statements will fail when the script is run unless
    it is started by a superuser (or the same user that owns all of the objects in the script).
    To make a script that can be restored by any user, but will give that user ownership of all
    the objects, specify -O.
    This option is only meaningful for the plain-text format. For the archive formats,
    you may specify the option when you call pg_restore.
-R
--no-reconnect
    This option is obsolete but still accepted for backwards compatibility.
-s
--schema-only

    Dump only the object definitions (schema), not data.
-S username
--superuser=username

    Specify the superuser user name to use when disabling triggers.
    This is only relevant if --disable-triggers is used. (Usually, it's better to leave this out,
    and instead start the resulting script as superuser.)
-t table
--table=table

      Dump data for table only. It is possible for there to be multiple tables with the same name
     in different schemas; if that is the case, all matching tables will be dumped.
    Specify both --schema and --table to select just one table.

        Note: In this mode, pg_dump makes no attempt to dump any other database
        objects that the selected table may depend upon. Therefore, there is no
        guarantee that the results of a single-table dump can be successfully restored
        by themselves into a clean database.
-v
--verbose

    Specifies verbose mode. This will cause pg_dump to output detailed object comments and
    start/stop times to the dump file, and progress messages to standard error.
-x
--no-privileges
--no-acl
    Prevent dumping of access privileges (grant/revoke commands).
-X disable-dollar-quoting
--disable-dollar-quoting
    This option disables the use of dollar quoting for function bodies, and forces them to be
    quoted using SQL standard string syntax.
-X disable-triggers
--disable-triggers

    This option is only relevant when creating a data-only dump. It instructs
    pg_dump to include commands to temporarily disable triggers on the target
    tables while the data is reloaded. Use this if you have referential integrity
    checks or other triggers on the tables that you do not want to invoke during data reload.
    Presently, the commands emitted for --disable-triggers must be done as superuser.
    So, you should also specify a superuser name with -S, or preferably be careful to
    start the resulting script as a superuser.
    This option is only meaningful for the plain-text format. For the archive formats,
    you may specify the option when you call pg_restore.
-X use-set-session-authorization
--use-set-session-authorization

    Output SQL-standard SET SESSION AUTHORIZATION commands instead of
    ALTER OWNER commands to determine object ownership. This makes the dump
    more standards compatible, but depending on the history of the objects in the dump,
    may not restore properly. Also, a dump using SET SESSION AUTHORIZATION will certainly
    require superuser privileges to restore correctly, whereas ALTER OWNER requires lesser privileges.
-Z 0..9
--compress=0..9
    Specify the compression level to use in archive formats that support compression.
    (Currently only the custom archive format supports compression.)
The following command-line options control the database connection parameters.
-h host
--host=host
    Specifies the host name of the machine on which the server is running.
    If the value begins with a slash, it is used as the directory for the Unix domain socket.
    The default is taken from the PGHOST environment variable, if set, else a
    Unix domain socket connection is attempted.
-p port
--port=port
    Specifies the TCP port or local Unix domain socket file extension on which the
    server is listening for connections. Defaults to the PGPORT environment variable, if set, or a compiled-in default.
-U username
    Connect as the given user
-W
    Force a password prompt. This should happen automatically if the server requires password authentication.

Environment

PGDATABASE
PGHOST
PGPORT
PGUSER

    

Default connection parameters.

Diagnostics

 pg_dump internally executes SELECT statements. If you have problems running pg_dump,

 make sure you are able to select information from the database using, for example, psql.
Notes

If your database cluster has any local additions to the template1 database, be careful to
restore the output of pg_dump into a truly empty database; otherwise you are likely
to get errors due to duplicate definitions of the added objects. To make an empty
database without any local additions, copy from template0 not template1, for example:

CREATE DATABASE foo WITH TEMPLATE template0;
pg_dump has a few limitations:


* When a data-only dump is chosen and the option --disable-triggers is used, pg_dump
* emits commands to disable triggers on user tables before inserting the data and
* commands to re-enable them after the data has been inserted. If the restore is
* stopped in the middle, the system catalogs may be left in the wrong state.

 

Members of tar archives are limited to a size less than 8 GB.
(This is an inherent limitation of the tar file format.) Therefore this format cannot be used if the
textual representation of any one table exceeds that size. The total size of a tar archive
and any of the other output formats is not limited, except possibly by the operating system.
 
The dump file produced by pg_dump does not contain the statistics used by the
optimizer to make query planning decisions. Therefore, it is wise to run ANALYZE
after restoring from a dump file to ensure good performance.

 

Because pg_dump is used to tranfer data to newer versions of PostgreSQL,
the output of pg_dump can be loaded into newer PostgreSQL databases.
It also can read older PostgreSQL databases. However, it usually cannot
read newer PostgreSQL databases or produce dump output that can be loaded
into older database versions. To do this, manual editing of the dump file might be required.

Examples

 To dump a database:
$ pg_dump mydb > db.out

 
To reload this database:
$ psql -d database -f db.out

To dump a database called mydb to a tar file:

$ pg_dump -Ft mydb > db.tar

To reload this dump into an existing database called newdb:

$ pg_restore -d newdb db.tar

pg_dumpall

Name
pg_dumpall -- extract a PostgreSQL database cluster into a script file
Synopsis

pg_dumpall [option...]
Description

 

pg_dumpall is a utility for writing out ("dumping") all PostgreSQL databases
of a cluster into one script file. The script file contains SQL commands
that can be used as input to psql to restore the databases.
It does this by calling pg_dump for each database in a cluster.
pg_dumpall also dumps global objects that are common to all databases.
(pg_dump does not save these objects.) This currently includes
information about database users and groups, and access permissions
that apply to databases as a whole.

Thus, pg_dumpall is an integrated solution for backing up your
databases. But note a limitation: it cannot dump "large objects",
since pg_dump cannot dump such objects into text files.
If you have databases containing large objects,
they should be dumped using one of pg_dump's non-text output modes.
 
Since pg_dumpall reads tables from all databases you will
most likely have to connect as a database superuser in order
to produce a complete dump. Also you will need superuser
privileges to execute the saved script in order to be allowed
to add users and groups, and to create databases.

The SQL script will be written to the standard output.
Shell operators should be used to redirect it into a file.

pg_dumpall needs to connect several times to the PostgreSQL
server (once per database). If you use password authentication
it is likely to ask for a password each time. It is convenient
to have a ~/.pgpass file in such cases. See Section 27.12 for more information.
Options

The following command-line options control the content and format of the output.

-a
--data-only

 Dump only the data, not the schema (data definitions).
-c
--clean

    Include SQL commands to clean (drop) the databases before recreating them.
-d
--inserts

    Dump data as INSERT commands (rather than COPY). This will make
    restoration very slow; it is mainly useful for making dumps that
    can be loaded into non-PostgreSQL databases. Note that the
    restore may fail altogether if you have rearranged column order.
    The -D option is safer, though even slower.
-D
--column-inserts
--attribute-inserts

     Dump data as INSERT commands with explicit column names
    (INSERT INTO table (column, ...) VALUES ...). This will make
    restoration very slow; it is mainly useful for making
    dumps that can be loaded into non-PostgreSQL databases.
-g
--globals-only

   Dump only global objects (users and groups), no databases.
-i
--ignore-version

    Ignore version mismatch between pg_dumpall and the database server.
    pg_dumpall can handle databases from previous releases of
    PostgreSQL, but very old versions are not supported anymore (
    currently prior to 7.0). Use this option if you need to
    override the version check (and if pg_dumpall then fails,
    don't say you weren't warned).
-o
--oids

    Dump object identifiers (OIDs) as part of the data for every
    table. Use this option if your application references the OID
    columns in some way (e.g., in a foreign key constraint).
    Otherwise, this option should not be used.
-O
--no-owner

      Do not output commands to set ownership of objects to
    match the original database. By default, pg_dumpall issues
    ALTER OWNER or SET SESSION AUTHORIZATION statements to set ownership
    of created schema elements. These statements will fail when
    the script is run unless it is started by a superuser (or the
    same user that owns all of the objects in the script).
    To make a script that can be restored by any user,
    but will give that user ownership of all the objects, specify -O.
-s
--schema-only

    Dump only the object definitions (schema), not data.
-S username
--superuser=username

    Specify the superuser user name to use when disabling triggers.
    This is only relevant if --disable-triggers is used. (Usually, it's
    better to leave this out, and instead start the
    resulting script as superuser.)
-v
--verbose

    Specifies verbose mode. This will cause pg_dumpall to
    output start/stop times to the dump file, and progress
    messages to standard error. It will also enable verbose
    output in pg_dump.
-x
--no-privileges
--no-acl

    Prevent dumping of access privileges (grant/revoke commands).
-X disable-dollar-quoting
--disable-dollar-quoting

    This option disables the use of dollar quoting for function

    bodies, and forces them to be quoted using SQL standard string syntax.
-X disable-triggers
--disable-triggers

    This option is only relevant when creating a data-only dump.
    It instructs pg_dumpall to include commands to temporarily
    disable triggers on the target tables while the data is
    reloaded. Use this if you have referential integrity
    checks or other triggers on the tables that you do not
    want to invoke during data reload.
    Presently, the commands emitted for --disable-triggers
    must be done as superuser. So, you should also
    specify a superuser name with -S, or preferably be

    careful to start the resulting script as a superuser.
-X use-set-session-authorization
--use-set-session-authorization

    Output SQL standard SET SESSION AUTHORIZATION commands
    instead of OWNER TO commands. This makes the dump more
    standards compatible, but depending on the history of the
    objects in the dump, may not restore properly.

The following command-line options control the database connection parameters.

-h host

    

    Specifies the host name of the machine on which the
    database server is running. If the value begins with a
    slash, it is used as the directory for the Unix domain
    socket. The default is taken from the PGHOST environment
    variable, if set, else a Unix domain socket connection
    is attempted.
-p port

    Specifies the TCP port or local Unix domain socket file
    extension on which the server is listening for connections.
    Defaults to the PGPORT environment variable, if set, or a compiled-in default.
-U username

   Connect as the given user.
-W

    Force a password prompt. This should happen automatically
    if the server requires password authentication.

Environment

PGHOST
PGPORT
PGUSER

Default connection parameters

Notes

Since pg_dumpall calls pg_dump internally, some diagnostic
messages will refer to pg_dump.

 

Once restored, it is wise to run ANALYZE on each database so the
optimizer has useful statistics. You can also run vacuumdb -a -z to
analyze all databases.
Examples

 To dump all databases:
$ pg_dumpall > db.out

To reload this database use, for example:

$ psql -f db.out template1

(It is not important to which database you connect here since the
script file created by pg_dumpall will contain the appropriate commands
to create and connect to the saved databases.)


 

FILE SYSTEM LEVEL BACKUP

An alternative backup strategy is to directly copy the files that
PostgreSQL uses to store the data in the database.

You can use whatever method you prefer for doing usual file
system backups, for example
 
tar -cf backup.tar /usr/local/pgsql/data

 
There are two restrictions, however, which make this method
impractical, or at least inferior to the pg_dump method:

 
The database server must be shut down in order to get a usable
backup. Half-way measures such as disallowing all connections

 
will not work (mainly because tar and similar tools do not
take an atomic snapshot of the state of the file system at a point in time).

Needless to say that you also need to shut down the server
before restoring the data.

Comments