Finding the status of running greenplum database system

posted Apr 28, 2017, 3:58 PM by Sachchida Ojha
gpstate utility provides the status of each individual component of a Greenplum Database system, including primary segments,mirror segments, master, and standby master. using $gpstate -e command you can find the state of the greenplum database system.

[gpadmin@sachi gpsne-1]$ gpstate -e
20131117:19:59:06:012754 gpstate:sachi:gpadmin-[INFO]:-Starting gpstate with args: -e
20131117:19:59:06:012754 gpstate:sachi:gpadmin-[INFO]:-local Greenplum Version: 'postgres (Greenplum Database) 4.2.2.4 build 1 Community Edition'
20131117:19:59:07:012754 gpstate:sachi:gpadmin-[INFO]:-master Greenplum Version: 'PostgreSQL 8.2.15 (Greenplum Database 4.2.2.4 build 1 Community Edition) on x86_64-unknown-linux-gnu, compiled by GCC gcc (GCC) 4.4.2 compiled on Oct 17 2012 11:52:28'
20131117:19:59:07:012754 gpstate:sachi:gpadmin-[INFO]:-Obtaining Segment details from master...
20131117:19:59:07:012754 gpstate:sachi:gpadmin-[INFO]:-Physical mirroring is not configured

[gpadmin@sachi gpsne-1]$ gpstate --help
COMMAND NAME: gpstate
Shows the status of a running Greenplum Database system.
*****************************************************
SYNOPSIS
*****************************************************
gpstate [-d <master_data_directory>] [-B <parallel_processes>] 
        [-s | -b | -Q | -e] [-m | -c] [-p] [-i] [-f] 
        [-v | -q] [-l <log_directory>]

gpstate -? | -h | --help
*****************************************************
DESCRIPTION
*****************************************************
The gpstate utility displays information about a running Greenplum Database instance. There is additional information you may want to know about a Greenplum Database system, since it is comprised of multiple PostgreSQL database instances (segments) spanning multiple machines. The gpstate utility provides 
additional status information for a Greenplum Database system, such as:
* Which segments are down.
* Master and segment configuration information (hosts, data directories, etc.).
* The ports used by the system.
* A mapping of primary segments to their corresponding mirror segments.
*****************************************************
OPTIONS
*****************************************************
-b (brief status)
Optional. Display a brief summary of the state of the Greenplum Database system. This is the default option.

-B <parallel_processes> 
The number of segments to check in parallel. If not specified,  the utility will start up to 60 parallel processes depending on how many segment instances it needs to check.

-c (show primary to mirror mappings)
Optional. Display mapping of primary segments to their corresponding mirror segments.

-d <master_data_directory>
Optional. The master data directory. If not specified, the value set for $MASTER_DATA_DIRECTORY will be used.

-e (show segments with mirror status issues)
Show details on primary/mirror segment pairs that have potential issues such as 1) the active segment is running in change tracking mode, meaning a segment is down 2) the active segment is in resynchronization mode, meaning it is catching up changes to the mirror 3) a segment is not in its preferred role, for example a segment that was a primary at system initialization time is now acting as a mirror, meaning you may have one or more segment hosts with unbalanced processing load.

-f (show standby master details)
Display details of the standby master host if configured.

-i (show Greenplum Database version)
Display the Greenplum Database software version information for each instance.

-l <logfile_directory>
The directory to write the log file. Defaults to ~/gpAdminLogs.

-m (list mirrors)
Optional. List the mirror segment instances in the system, their current role, and synchronization status.

-p (show ports)
List the port numbers used throughout the Greenplum Database system.

-q (no screen output)
Optional. Run in quiet mode. Except for warning messages, command output is not displayed on the screen. However, this information is still written to the log file.

-Q (quick status)
Optional. Checks segment status in the system catalog on the master host. Does not poll the segments for status.

-s (detailed status)
Optional. Displays detailed status information for the Greenplum Database system.

-v (verbose output)
Optional. Displays error messages and outputs detailed status and progress information.

-? | -h | --help (help)
 Displays the online help.

*****************************************************
OUTPUT DEFINITIONS FOR DETAIL VIEW
*****************************************************
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
MASTER OUTPUT DATA
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

* Master host - host name of the master
* Master postgres process ID - PID of the master postgres database listener process
* Master data directory - file system location of the master data directory
* Master port - port of the master database listener process
* Master current role - dispatch = regular operating mode utility = maintenance mode 
* Greenplum array configuration type - Standard = one NIC per host Multi-Home = multiple NICs per host
* Greenplum initsystem version - version of Greenplum Database when system was first initialized
* Greenplum current version - current version of Greenplum Database
* Postgres version - version of PostgreSQL that Greenplum Database is based on
* Greenplum mirroring status - physical mirroring, SAN or none
* Master standby - host name of the standby master
* Standby master state - status of the standby master: active or passive

+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
SEGMENT OUTPUT DATA
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
* Hostname - system-configured host name
* Address - network address host name (NIC name)
* Datadir - file system location of segment data directory
* Port - port number of segment postgres database listener process
* Current Role - current role of a segment: Mirror or Primary 
* Preferred Role - role at system initialization time: Mirror or Primary
* Mirror Status - status of a primary/mirror segment pair:
Synchronized = data is up to date on both 
Resynchronizing = data is currently being copied from one to the other
Change Tracking = segment down and active segment is logging changes
* Change tracking data size - when in Change Tracking mode, the size of the change log file (may grow and shrink as compression is applied)
* Estimated total data to synchronize - when in Resynchronization mode, the estimated size of data left to syncronize
* Data synchronized - when in Resynchronization mode, the estimated size of data that has already been syncronized
* Estimated resync progress with mirror - when in Resynchronization mode, the estimated percentage of completion
* Estimated resync end time - when in Resynchronization mode, the estimated time to complete
* File postmaster.pid - status of postmaster.pid lock file: Found or Missing
* PID from postmaster.pid file - PID found in the postmaster.pid file
* Lock files in /tmp - a segment port lock file for its postgres process is created in /tmp (file is removed when a segment shuts down)
* Active PID - active process ID of a segment
* Master reports status as - segment status as reported in the system catalog:  Up or Down
Database status - status of Greenplum Database to incoming requests: Up, Down, or Suspended. A Suspended state means database activity is temporarily paused while a segment transitions from one state to another.
*****************************************************
EXAMPLES
*****************************************************
Show detailed status information of a Greenplum Database system:
gpstate -s
$ gpstate -s
20131117:20:10:29:014187 gpstate:sachi:gpadmin-[INFO]:-Starting gpstate with args: -s
20131117:20:10:29:014187 gpstate:sachi:gpadmin-[INFO]:-local Greenplum Version: 'postgres (Greenplum Database) 4.2.2.4 build 1 Community Edition'
20131117:20:10:29:014187 gpstate:sachi:gpadmin-[INFO]:-master Greenplum Version: 'PostgreSQL 8.2.15 (Greenplum Database 4.2.2.4 build 1 Community Edition) on x86_64-unknown-linux-gnu, compiled by GCC gcc (GCC) 4.4.2 compiled on Oct 17 2012 11:52:28'
20131117:20:10:29:014187 gpstate:sachi:gpadmin-[INFO]:-Obtaining Segment details from master...
20131117:20:10:29:014187 gpstate:sachi:gpadmin-[INFO]:-Gathering data from segments...
.. 
20131117:20:10:32:014187 gpstate:sachi:gpadmin-[INFO]:-----------------------------------------------------
20131117:20:10:32:014187 gpstate:sachi:gpadmin-[INFO]:--Master Configuration & Status
20131117:20:10:32:014187 gpstate:sachi:gpadmin-[INFO]:-----------------------------------------------------
20131117:20:10:32:014187 gpstate:sachi:gpadmin-[INFO]:-   Master host                    = sachi
20131117:20:10:32:014187 gpstate:sachi:gpadmin-[INFO]:-   Master postgres process ID     = 5307
20131117:20:10:32:014187 gpstate:sachi:gpadmin-[INFO]:-   Master data directory          = /home/gpmaster/gpsne-1
20131117:20:10:32:014187 gpstate:sachi:gpadmin-[INFO]:-   Master port                    = 5432
20131117:20:10:32:014187 gpstate:sachi:gpadmin-[INFO]:-   Master current role            = dispatch
20131117:20:10:32:014187 gpstate:sachi:gpadmin-[INFO]:-   Greenplum initsystem version   = 4.2.2.4 build 1 Community Edition
20131117:20:10:32:014187 gpstate:sachi:gpadmin-[INFO]:-   Greenplum current version      = PostgreSQL 8.2.15 (Greenplum Database 4.2.2.4 build 1 Community Edition) on x86_64-unknown-linux-gnu, compiled by GCC gcc (GCC) 4.4.2 compiled on Oct 17 2012 11:52:28
20131117:20:10:32:014187 gpstate:sachi:gpadmin-[INFO]:-   Postgres version               = 8.2.15
20131117:20:10:32:014187 gpstate:sachi:gpadmin-[INFO]:-   Master standby                 = No master standby configured
20131117:20:10:32:014187 gpstate:sachi:gpadmin-[INFO]:-----------------------------------------------------
20131117:20:10:32:014187 gpstate:sachi:gpadmin-[INFO]:-Segment Instance Status Report
20131117:20:10:32:014187 gpstate:sachi:gpadmin-[INFO]:-----------------------------------------------------
20131117:20:10:32:014187 gpstate:sachi:gpadmin-[INFO]:-   Segment Info
20131117:20:10:32:014187 gpstate:sachi:gpadmin-[INFO]:-      Hostname                          = sachi
20131117:20:10:32:014187 gpstate:sachi:gpadmin-[INFO]:-      Address                           = sachi
20131117:20:10:32:014187 gpstate:sachi:gpadmin-[INFO]:-      Datadir                           = /disk1/gpdata1/gpsne0
20131117:20:10:32:014187 gpstate:sachi:gpadmin-[INFO]:-      Port                              = 40000
20131117:20:10:32:014187 gpstate:sachi:gpadmin-[INFO]:-   Status
20131117:20:10:32:014187 gpstate:sachi:gpadmin-[INFO]:-      PID                               = 5275
20131117:20:10:32:014187 gpstate:sachi:gpadmin-[INFO]:-      Configuration reports status as   = Up
20131117:20:10:32:014187 gpstate:sachi:gpadmin-[INFO]:-      Database status                   = Up
20131117:20:10:32:014187 gpstate:sachi:gpadmin-[INFO]:-----------------------------------------------------
20131117:20:10:32:014187 gpstate:sachi:gpadmin-[INFO]:-   Segment Info
20131117:20:10:32:014187 gpstate:sachi:gpadmin-[INFO]:-      Hostname                          = sachi
20131117:20:10:32:014187 gpstate:sachi:gpadmin-[INFO]:-      Address                           = sachi
20131117:20:10:32:014187 gpstate:sachi:gpadmin-[INFO]:-      Datadir                           = /disk2/gpdata2/gpsne1
20131117:20:10:32:014187 gpstate:sachi:gpadmin-[INFO]:-      Port                              = 40001
20131117:20:10:32:014187 gpstate:sachi:gpadmin-[INFO]:-   Status
20131117:20:10:32:014187 gpstate:sachi:gpadmin-[INFO]:-      PID                               = 5276
20131117:20:10:32:014187 gpstate:sachi:gpadmin-[INFO]:-      Configuration reports status as   = Up
20131117:20:10:32:014187 gpstate:sachi:gpadmin-[INFO]:-      Database status                   = Up
[gpadmin@sachi gpsne-1]$ 


Do a quick check for down segments in the master host system catalog:
gpstate -Q
[gpadmin@sachi gpsne-1]$ gpstate -Q
20131117:20:11:35:014481 gpstate:sachi:gpadmin-[INFO]:-Starting gpstate with args: -Q
20131117:20:11:35:014481 gpstate:sachi:gpadmin-[INFO]:-local Greenplum Version: 'postgres (Greenplum Database) 4.2.2.4 build 1 Community Edition'
20131117:20:11:35:014481 gpstate:sachi:gpadmin-[INFO]:-master Greenplum Version: 'PostgreSQL 8.2.15 (Greenplum Database 4.2.2.4 build 1 Community Edition) on x86_64-unknown-linux-gnu, compiled by GCC gcc (GCC) 4.4.2 compiled on Oct 17 2012 11:52:28'
20131117:20:11:35:014481 gpstate:sachi:gpadmin-[INFO]:-Obtaining Segment details from master...
20131117:20:11:35:014481 gpstate:sachi:gpadmin-[INFO]:--Quick Greenplum database status from Master instance only
20131117:20:11:35:014481 gpstate:sachi:gpadmin-[INFO]:-----------------------------------------------------------
20131117:20:11:35:014481 gpstate:sachi:gpadmin-[INFO]:-# of up segments, from configuration table     = 2
20131117:20:11:35:014481 gpstate:sachi:gpadmin-[INFO]:-----------------------------------------------------------


Show information about mirror segment instances:
gpstate -m
[gpadmin@sachi gpsne-1]$ gpstate -m
20131117:20:12:02:014537 gpstate:sachi:gpadmin-[INFO]:-Starting gpstate with args: -m
20131117:20:12:02:014537 gpstate:sachi:gpadmin-[INFO]:-local Greenplum Version: 'postgres (Greenplum Database) 4.2.2.4 build 1 Community Edition'
20131117:20:12:02:014537 gpstate:sachi:gpadmin-[INFO]:-master Greenplum Version: 'PostgreSQL 8.2.15 (Greenplum Database 4.2.2.4 build 1 Community Edition) on x86_64-unknown-linux-gnu, compiled by GCC gcc (GCC) 4.4.2 compiled on Oct 17 2012 11:52:28'
20131117:20:12:02:014537 gpstate:sachi:gpadmin-[INFO]:-Obtaining Segment details from master...
20131117:20:12:02:014537 gpstate:sachi:gpadmin-[WARNING]:--------------------------------------------------------------
20131117:20:12:02:014537 gpstate:sachi:gpadmin-[WARNING]:-physical mirroring not used
20131117:20:12:02:014537 gpstate:sachi:gpadmin-[WARNING]:--------------------------------------------------------------

Show information about the standby master configuration:
gpstate -f
gpadmin@sachi gpsne-1]$ gpstate -f
20131117:20:12:22:014600 gpstate:sachi:gpadmin-[INFO]:-Starting gpstate with args: -f
20131117:20:12:22:014600 gpstate:sachi:gpadmin-[INFO]:-local Greenplum Version: 'postgres (Greenplum Database) 4.2.2.4 build 1 Community Edition'
20131117:20:12:22:014600 gpstate:sachi:gpadmin-[INFO]:-master Greenplum Version: 'PostgreSQL 8.2.15 (Greenplum Database 4.2.2.4 build 1 Community Edition) on x86_64-unknown-linux-gnu, compiled by GCC gcc (GCC) 4.4.2 compiled on Oct 17 2012 11:52:28'
20131117:20:12:22:014600 gpstate:sachi:gpadmin-[INFO]:-Obtaining Segment details from master...
20131117:20:12:22:014600 gpstate:sachi:gpadmin-[INFO]:-Standby master instance not configured
20131117:20:12:22:014600 gpstate:sachi:gpadmin-[INFO]:--------------------------------------------------------------
20131117:20:12:22:014600 gpstate:sachi:gpadmin-[INFO]:--gp_master_mirroring table
20131117:20:12:22:014600 gpstate:sachi:gpadmin-[INFO]:--------------------------------------------------------------
20131117:20:12:22:014600 gpstate:sachi:gpadmin-[INFO]:--Summary state: Not Configured
20131117:20:12:22:014600 gpstate:sachi:gpadmin-[INFO]:--Detail state: 
20131117:20:12:22:014600 gpstate:sachi:gpadmin-[INFO]:--Log time: 
20131117:20:12:22:014600 gpstate:sachi:gpadmin-[INFO]:--------------------------------------------------------------


Display the Greenplum software version information:
gpstate -i
[gpadmin@sachi gpsne-1]$ gpstate -i
20131117:20:12:45:014658 gpstate:sachi:gpadmin-[INFO]:-Starting gpstate with args: -i
20131117:20:12:45:014658 gpstate:sachi:gpadmin-[INFO]:-local Greenplum Version: 'postgres (Greenplum Database) 4.2.2.4 build 1 Community Edition'
20131117:20:12:46:014658 gpstate:sachi:gpadmin-[INFO]:-master Greenplum Version: 'PostgreSQL 8.2.15 (Greenplum Database 4.2.2.4 build 1 Community Edition) on x86_64-unknown-linux-gnu, compiled by GCC gcc (GCC) 4.4.2 compiled on Oct 17 2012 11:52:28'
20131117:20:12:46:014658 gpstate:sachi:gpadmin-[INFO]:-Obtaining Segment details from master...
20131117:20:12:46:014658 gpstate:sachi:gpadmin-[INFO]:-Loading version information
20131117:20:12:47:014658 gpstate:sachi:gpadmin-[INFO]:-   Host    Datadir                  Port    Version                                                                                                                                                                  
20131117:20:12:47:014658 gpstate:sachi:gpadmin-[INFO]:-   sachi   /home/gpmaster/gpsne-1   5432    PostgreSQL 8.2.15 (Greenplum Database 4.2.2.4 build 1 Community Edition) on x86_64-unknown-linux-gnu, compiled by GCC gcc (GCC) 4.4.2 compiled on Oct 17 2012 11:52:28
20131117:20:12:47:014658 gpstate:sachi:gpadmin-[INFO]:-   sachi   /disk1/gpdata1/gpsne0    40000   PostgreSQL 8.2.15 (Greenplum Database 4.2.2.4 build 1 Community Edition) on x86_64-unknown-linux-gnu, compiled by GCC gcc (GCC) 4.4.2 compiled on Oct 17 2012 11:52:28
20131117:20:12:47:014658 gpstate:sachi:gpadmin-[INFO]:-   sachi   /disk2/gpdata2/gpsne1    40001   PostgreSQL 8.2.15 (Greenplum Database 4.2.2.4 build 1 Community Edition) on x86_64-unknown-linux-gnu, compiled by GCC gcc (GCC) 4.4.2 compiled on Oct 17 2012 11:52:28
20131117:20:12:47:014658 gpstate:sachi:gpadmin-[INFO]:-All segments are running the same software version
Comments