Prompt Engineering Reference Hub — Enterprise Matrix Author: William J. Lawrence

Master Index Hub Overview

Welcome to the Convoluted Organization™ Enterprise Reference Hub. Below are all 38 integrated infrastructure platforms, fully color-coded by category type, expanding each reference table to 200 exhaustive administrative commands with a verbose technical summary header.

Microsoft Purview Gov

Unified data governance, lineage tracking, and automated asset scanning.

Collibra Governance Gov

REST API and administrative reference matrix for Collibra governance.

Alation Catalog Sync Gov

Metadata extraction, stewardship metrics, and bulk sync matrix.

PostgreSQL OLTP

Core transactional engine parameters, extensions, and CLI maintenance.

MySQL Enterprise OLTP

InnoDB configurations, replication setups, and operational commands.

SQL Server T-SQL OLTP

Administrative, diagnostic, and maintenance script matrix for T-SQL.

Oracle Database OLTP

SQL*Plus, RMAN operations, and tablespace management matrix.

IBM Db2 z/OS OLTP

TSO/ISPF console commands and subsystem operations matrix.

MariaDB OLTP

Aria/InnoDB storage engine and administrative CLI matrix.

SQLite OLTP

Lightweight CLI, PRAGMA configuration, and administrative matrix.

CockroachDB OLTP

Distributed SQL, multi-node ranges, and cluster management.

PingCAP TiDB OLTP

Distributed HTAP, Placement Driver, and TiUP operational matrix.

Amazon Aurora OLTP

AWS CLI, RDS parameter groups, and managed cluster operations.

Snowflake Data Cloud OLAP

Virtual warehouse sizing, RBAC, time travel, and stream-tasks.

Google BigQuery OLAP

Serverless data warehouse bq CLI and analytics SQL matrix.

Amazon Redshift OLAP

AWS CLI and SQL administrative columnar warehouse matrix.

ClickHouse Analytics OLAP

MergeTree columnar engine and high-speed analytical queries.

Databricks Unity OLAP

Unity catalog governance, Delta Lake optimization, and Spark SQL.

Apache Druid OLAP

Real-time analytical time-series and ingestion supervisors.

DuckDB OLAP

Embedded vectorized analytical processing and cloud extension CLI.

Trino (Presto) OLAP

Federated distributed SQL querying across multi-source catalogs.

Apache Hive OLAP

Beeline CLI, Metastore catalog, and HiveQL operational matrix.

Apache Iceberg OLAP

Open table format snapshot history, compaction, and maintenance.

MongoDB Enterprise NOSQL

Document-oriented database, replica sets, and aggregations.

Apache Cassandra NOSQL

Wide-column distributed cqlsh CLI and nodetool maintenance.

Apache HBase NOSQL

Hadoop column family database shell and cluster administration.

CouchDB NOSQL

Document store REST API, view indexing, and Couch replication.

Amazon DynamoDB NOSQL

Serverless NoSQL key-value store AWS CLI administration.

Redis Enterprise NOSQL

In-memory data structure store, caching, and streams matrix.

Memcached NOSQL

Distributed memory object caching server management matrix.

Apache Flink Stream

Stateful stream processing and real-time pipeline management.

Apache Kafka Events Stream

Distributed event streams, topics, and consumer groups.

InfluxDB Stream

Time-series database metrics, Flux queries, and buckets matrix.

Neo4j Graph GRAPH

Connected data traversal, Cypher querying, and graph algorithms.

Amazon Neptune Graph GRAPH

Managed graph cluster, Gremlin traversal, and SPARQL matrix.

Milvus Vector DB VECTOR

High-performance vector similarity search and index management.

Qdrant Vector DB VECTOR

Vector database similarity search, payload filters, and snapshots.

Pinecone Vector Index VECTOR

Managed vector index serverless SDK and namespace operations.

Microsoft Purview Gov

Technical Architecture & Overview

Microsoft Purview is an enterprise-grade, unified data governance and compliance solution designed to manage and govern data across on-premises, multi-cloud (Azure, AWS, GCP), and SaaS environments. Built on an open Apache Atlas metadata framework, Purview automatically scans, classifies, and maps data assets to construct an end-to-end data map and automated lineage graph.

Primary Use Cases: Automated data discovery, PII/PHI sensitive data classification, automated data lineage tracking for ETL/ELT pipelines (Data Factory, Synapse, Databricks), business glossary management, and fine-grained attribute-based access control (ABAC) policies across distributed enterprise storage.

Core Components: Data Map (Atlas-compatible graph engine), Data Catalog (search, business glossary, asset curation), Data Insights (operational telemetry, classification coverage, sensitivity label distributions), and Policy Store (access policies and consent enforcement).

Exhaustive 200-command administrative, diagnostic, and operational matrix for Microsoft Purview.

#Operation / ActionAPI / PowerShell SyntaxDescription
1Get Account InfoGet-AzPurviewAccount -Name 'org-purview'Retrieves Purview account metadata.
2New Scan RuleNew-AzPurviewScanRule -Endpoint https://org-purview.scan.purview.azure.comDefines custom classification rules.
3Trigger Data ScanStart-AzPurviewScan -ScanName 'SqlScan' -ResourceGroupName 'rg-data'Initiates source metadata extraction.
4List ClassificationsGet-AzPurviewClassificationRuleOutputs built-in/custom classifiers.
5Get Asset LineageGET /catalog/api/atlas/v2/lineage/{guid}Extracts end-to-end data provenance.
6Search Catalog AssetsPOST /catalog/api/search/queryExecutes full-text asset search.
7Create Glossary TermPOST /catalog/api/glossary/termsAdds a governance business term.
8List CollectionsGET /account/api/collectionsRetrieves hierarchical collection nodes.
9Assign Collection RolePOST /account/api/collections/{name}/roleAssignmentsDelegates security roles.
10Get Scan StatusGet-AzPurviewScanStatus -ScanId 'scan-123'Inspects active scan progress.
11Delete Asset RecordDELETE /catalog/api/atlas/v2/entity/guid/{guid}Purges metadata entity from catalog.
12Create DataSourcePUT /scan/api/datasources/{dataSourceName}Registers a new target data source.
13List Data SourcesGET /scan/api/datasourcesLists all registered data sources.
14Get Classification RulesGET /scan/api/classificationrulesFetches all active scanning rules.
15Trigger Bulk ClassificationPOST /catalog/api/discovery/classifyRuns batch asset classification.
16Export Sensitivity LabelsGet-AzPurviewSensitivityLabelOutputs applied sensitivity tags.
17Create Policy AssignmentPOST /policy/api/assignmentsApplies attribute-based access policy.
18Get Insight MetricsGET /insights/api/asset/countRetrieves asset catalog volume counts.
19Set Managed EndpointSet-AzPurviewManagedPrivateEndpointConfigures secure network peering.
20Get Insights ScansGET /insights/api/scan/statusFetches operational scan analytics.
21List Custom AttributesGET /catalog/api/types/typedefs?type=entityInspects metadata schema definitions.
22Create Business MetadataPOST /catalog/api/types/typedefsAdds custom metadata attributes.
23Get Credential ReferenceGet-AzPurviewCredentialRetrieves secure connector key mapping.
24Create CredentialNew-AzPurviewCredential -Name 'SQLAuth'Provisions a new scanning credential.
25Get Kafka NotificationGET /notifications/api/eventsRetrieves event stream hooks.
26Update Glossary TermPUT /catalog/api/glossary/terms/{guid}Modifies business definition tags.
27Get Policy StatusGET /policy/api/statusVerifies data plane policy deployment.
28List Kerberos ConfigurationsGET /scan/api/kerberosInspects local enterprise auth setups.
29Test Data Source ConnectivityPOST /scan/api/datasources/{name}/testConnectValidates source reachability.
30Remove Purview AccountRemove-AzPurviewAccount -Name 'org-purview'Decommissions the Purview service.
31Get Entity by GUIDGET /catalog/api/atlas/v2/entity/guid/{guid}Fetches details of a specific metadata entity.
32Bulk Create EntitiesPOST /catalog/api/atlas/v2/entity/bulkCreates multiple metadata entities in one payload.
33Bulk Update EntitiesPOST /catalog/api/atlas/v2/entity/bulkUpdates multiple entity properties simultaneously.
34Delete Entity by Unique AttributeDELETE /catalog/api/atlas/v2/entity/uniqueAttribute/type/{typeName}Deletes an entity based on a unique attribute.
35Get Glossary by GUIDGET /catalog/api/glossary/{glossaryGuid}Retrieves a business glossary by its unique GUID.
36Create New GlossaryPOST /catalog/api/glossaryEstablishes a new top-level business glossary.
37Delete GlossaryDELETE /catalog/api/glossary/{glossaryGuid}Removes a business glossary and its terms.
38Get Term by GUIDGET /catalog/api/glossary/term/{termGuid}Fetches detailed metadata for a glossary term.
39Import Glossary TermsPOST /catalog/api/glossary/{glossaryGuid}/terms/importBulk imports business terms from CSV.
40Export Glossary TermsPOST /catalog/api/glossary/{glossaryGuid}/terms/exportExports business terms to downloadable CSV.
41Get Term Assigned EntitiesGET /catalog/api/glossary/terms/{termGuid}/assignedEntitiesLists all catalog assets assigned to a term.
42Assign Term to EntityPOST /catalog/api/glossary/terms/{termGuid}/assignedEntitiesLinks a business term to catalog assets.
43Disassociate Term from EntityDELETE /catalog/api/glossary/terms/{termGuid}/assignedEntitiesUnlinks a business term from catalog assets.
44Get Collection PathGET /account/api/collections/{collectionName}Retrieves hierarchical path of a collection.
45Create Sub-CollectionPUT /account/api/collections/{parentName}/childCollections/{childName}Provisions a nested collection under parent.
46Delete CollectionDELETE /account/api/collections/{collectionName}Removes a governance collection node.
47List Collection Role AssignmentsGET /account/api/collections/{collectionName}/roleAssignmentsInspects security roles assigned to a collection.
48Revoke Collection Role AssignmentDELETE /account/api/collections/{collectionName}/roleAssignments/{roleAssignmentId}Revokes security access from a collection.
49Get Key Vault ConnectionGET /scan/api/keyvaults/{keyVaultName}Inspects Azure Key Vault connector settings.
50Register Key VaultPUT /scan/api/keyvaults/{keyVaultName}Registers Key Vault for scanning secret management.
51List Scans for Data SourceGET /scan/api/datasources/{dataSourceName}/scansLists all scan configurations for a data source.
52Create Scan ConfigurationPUT /scan/api/datasources/{dataSourceName}/scans/{scanName}Provisions a new scan schedule and rule set.
53Run Immediate ScanPOST /scan/api/datasources/{dataSourceName}/scans/{scanName}/runTriggers an on-demand scan execution.
54Cancel Running ScanPOST /scan/api/datasources/{dataSourceName}/scans/{scanName}/runs/{runId}/cancelStops an active data scanning job.
55Get Scan HistoryGET /scan/api/datasources/{dataSourceName}/scans/{scanName}/runsFetches execution log history for scans.
56Get Scan FilterGET /scan/api/datasources/{dataSourceName}/scans/{scanName}/filters/customInspects asset inclusion/exclusion filters.
57Update Scan FilterPUT /scan/api/datasources/{dataSourceName}/scans/{scanName}/filters/customUpdates file and path filters for scanning.
58List System Classification RulesGET /scan/api/systemClassificationRulesLists default system-defined classifiers.
59Create Custom Classification RulePUT /scan/api/classificationrules/{ruleName}Creates custom regex or dictionary classifier.
60Delete Classification RuleDELETE /scan/api/classificationrules/{ruleName}Removes a custom classification rule.
61List Rule SetsGET /scan/api/scanrulesetsLists all system and custom scan rule sets.
62Create Scan Rule SetPUT /scan/api/scanrulesets/{setName}Groups classifiers into a scan rule set.
63Delete Scan Rule SetDELETE /scan/api/scanrulesets/{setName}Removes a custom scan rule set.
64Get Lineage EdgeGET /catalog/api/atlas/v2/lineage/{guid}/nextRetrieves downstream data flow lineage steps.
65Get Ancestor LineageGET /catalog/api/atlas/v2/lineage/{guid}/previousRetrieves upstream data flow origin steps.
66Create Custom RelationshipPOST /catalog/api/atlas/v2/relationshipCreates custom relationship between assets.
67Delete RelationshipDELETE /catalog/api/atlas/v2/relationship/guid/{guid}Removes a relationship link between assets.
68Get Type DefinitionGET /catalog/api/atlas/v2/types/typedef/name/{name}Inspects structural schema for type definition.
69Delete Type DefinitionDELETE /catalog/api/atlas/v2/types/typedefsDeletes custom entity or attribute definitions.
70List All ClassificationsGET /catalog/api/atlas/v2/types/typedefs?type=classificationLists all classification type definitions.
71Create Custom Classification TypePOST /catalog/api/atlas/v2/types/typedefsDefines new custom classification category.
72Add Label to EntityPOST /catalog/api/atlas/v2/entity/guid/{guid}/labelsApplies tags/labels to a catalog asset.
73Remove Label from EntityDELETE /catalog/api/atlas/v2/entity/guid/{guid}/labelsRemoves tags/labels from a catalog asset.
74List Sensitivity LabelsGET /policy/api/sensitivitylabelsLists Purview Microsoft Information Protection labels.
75Set Asset Sensitivity LabelPOST /catalog/api/entity/guid/{guid}/sensitivitylabelApplies MIP sensitivity label to asset.
76Get Data Map MetricsGET /insights/api/datamap/summaryRetrieves data map asset counts and size metrics.
77Get Governance InsightsGET /insights/api/governance/coverageEvaluates stewardship coverage across data map.
78Get Classification InsightsGET /insights/api/classifications/summaryRetrieves breakdown of applied classifications.
79Get Sensitivity InsightsGET /insights/api/sensitivity/summaryRetrieves breakdown of MIP sensitivity labels.
80Get Glossary InsightsGET /insights/api/glossary/summaryEvaluates glossary term adoption and mapping.
81List Policy Store RulesGET /policy/api/policystore/policiesLists data governance access policies.
82Create Data Access PolicyPOST /policy/api/policystore/policiesProvisions access rule for Purview data map.
83Delete Access PolicyDELETE /policy/api/policystore/policies/{policyId}Removes data access policy from store.
84Set Account FirewallSet-AzPurviewAccount -Name 'org-purview' -PublicNetworkAccess DisabledEnforces private network perimeter.
85Create Private EndpointNew-AzPurviewPrivateEndpointConnectionProvisions private endpoint connection.
86Approve Private EndpointApprove-AzPurviewPrivateEndpointConnectionApproves pending private link request.
87Reject Private EndpointDeny-AzPurviewPrivateEndpointConnectionRejects private link connection request.
88List Private EndpointsGet-AzPurviewPrivateEndpointConnectionLists all connected private endpoints.
89Set Resource TagsUpdate-AzPurviewAccount -Name 'org-purview' -Tag @{Env='Prod'}Applies Azure tags to Purview resource.
90Get Event Hub Notification ConfigGET /account/api/eventhub/notificationInspects Kafka/Event Hub streaming hooks.
91Set Event Hub Notification ConfigPUT /account/api/eventhub/notificationConfigures streaming diagnostic notification hub.
92Get Purview SKU InfoGet-AzPurviewAccountSkuInspects throughput capacity and SKU units.
93Update Purview SKUUpdate-AzPurviewAccount -Name 'org-purview' -SkuCapacity 4Scales Purview Data Map capacity units.
94List Managed ResourcesGet-AzPurviewManagedResourceLists storage accounts managed by Purview.
95Get Scan DiagnosticsGET /scan/api/diagnostics/summaryRetrieves error logs for failed scan jobs.
96Purge Catalog CachePOST /catalog/api/cache/purgeFlushes temporary search and query caches.
97Verify Metastore SyncGET /catalog/api/metastore/sync/statusChecks metadata synchronization state.
98List Business Metadata TypesGET /catalog/api/types/typedefs?type=business_metadataLists custom business metadata groups.
99Assign Business MetadataPOST /catalog/api/atlas/v2/entity/guid/{guid}/businessmetadataApplies business metadata attributes to asset.
100Remove Business MetadataDELETE /catalog/api/atlas/v2/entity/guid/{guid}/businessmetadataRemoves business metadata attributes from asset.
101Get Private Link ResourceGet-AzPurviewPrivateLinkResource -AccountName 'org-purview'Lists supported private link resource types.
102New Private Link ServiceNew-AzPurviewPrivateEndpointConnectionProvisions a private endpoint for secure catalog traffic.
103Get Account Key Vault LinkGet-AzPurviewKeyVault -AccountName 'org-purview'Retrieves Key Vault integration references.
104Add Key Vault ConnectionNew-AzPurviewKeyVault -AccountName 'org-purview' -KeyVaultUrl 'https://kv.vault.azure.net/'Registers Key Vault for scanning secret management.
105Remove Key Vault ConnectionRemove-AzPurviewKeyVault -AccountName 'org-purview' -KeyVaultName 'kv'Deletes Key Vault integration from Purview.
106Get Data Factory Lineage SyncGET /catalog/api/atlas/v2/lineage/datafactory/{factoryGuid}Inspects Azure Data Factory pipeline lineage sync status.
107Get Databricks Lineage SyncGET /catalog/api/atlas/v2/lineage/databricks/{workspaceId}Inspects Unity Catalog to Purview lineage connector.
108Get Synapse Lineage SyncGET /catalog/api/atlas/v2/lineage/synapse/{workspaceGuid}Inspects Azure Synapse Analytics pipeline lineage sync.
109Get Snowflake Lineage SyncGET /catalog/api/atlas/v2/lineage/snowflake/{accountName}Inspects Snowflake data map lineage extraction.
110Get PowerBI Lineage SyncGET /catalog/api/atlas/v2/lineage/powerbi/{tenantId}Inspects PowerBI dashboard and dataset lineage flow.
111Get SQL Server Lineage SyncGET /catalog/api/atlas/v2/lineage/sqlserver/{serverGuid}Inspects SQL Server stored procedure lineage sync.
112Create Custom Atlas TypePOST /catalog/api/atlas/v2/types/typedefsProvisions custom COM_Asset type in Atlas graph.
113Get Entity Lineage DepthGET /catalog/api/atlas/v2/lineage/{guid}?direction=BOTH&depth=5Fetches 5-level upstream and downstream lineage graph.
114Get Lineage Impact AnalysisGET /catalog/api/atlas/v2/lineage/{guid}/impactEvaluates downstream impact score for asset changes.
115Set Sensitivity Label BatchPOST /catalog/api/atlas/v2/entity/bulk/classificationApplies sensitivity classification to batch of GUIDs.
116Remove Sensitivity Label BatchDELETE /catalog/api/atlas/v2/entity/bulk/classificationRemoves sensitivity tags from batch of GUIDs.
117Get Data Map Capacity UnitsGet-AzPurviewAccount -Name 'org-purview' | Select-Object SkuCapacityInspects provisioned capacity units (4 CUs = 10k assets/sec).
118Set Data Map Auto ScaleSet-AzPurviewAccount -Name 'org-purview' -SkuCapacity 16Scales Purview Data Map to 16 CUs for large ingestion.
119Export Data Map Entities CSVPOST /catalog/api/atlas/v2/entity/exportExports asset catalog to downloadable CSV archive.
120Import Data Map Entities CSVPOST /catalog/api/atlas/v2/entity/importImports external metadata assets from CSV payload.
121Get Atlas Business Metadata GroupsGET /catalog/api/atlas/v2/types/typedefs?type=business_metadataLists all custom business metadata groups.
122Get Business Metadata Group SchemaGET /catalog/api/atlas/v2/types/typedef/name/{groupName}Inspects schema of custom business metadata group.
123Delete Business Metadata GroupDELETE /catalog/api/atlas/v2/types/typedefsDeletes custom business metadata group definition.
124Get Term Parent HierarchyGET /catalog/api/glossary/terms/{termGuid}/parentsRetrieves parent term taxonomy nodes.
125Get Term Child HierarchyGET /catalog/api/glossary/terms/{termGuid}/childrenRetrieves child term taxonomy nodes.
126Link Related TermsPOST /catalog/api/glossary/terms/{termGuid}/relatedCreates bidirectional relationship between business terms.
127Unlink Related TermsDELETE /catalog/api/glossary/terms/{termGuid}/relatedRemoves relationship link between business terms.
128Get Term SynonymsGET /catalog/api/glossary/terms/{termGuid}/synonymsLists alternate synonym names for business term.
129Add Term SynonymPOST /catalog/api/glossary/terms/{termGuid}/synonymsAdds alternate synonym to business term.
130Get Term AcronymsGET /catalog/api/glossary/terms/{termGuid}/acronymsLists business acronyms mapped to term.
131Add Term AcronymPOST /catalog/api/glossary/terms/{termGuid}/acronymsAdds business acronym mapping to term.
132Get Collection Hierarchy TreeGET /account/api/collections/treeFetches complete organizational collection tree.
133Move Asset Between CollectionsPOST /catalog/api/atlas/v2/entity/guid/{guid}/move?targetCollection={collectionName}Reassigns asset to new collection node.
134Bulk Move Assets Between CollectionsPOST /catalog/api/atlas/v2/entity/bulk/moveBatch reassigns assets to new collection node.
135Get Collection StatsGET /account/api/collections/{collectionName}/statsCalculates total sub-collections and assets in collection.
136List Self-Hosted Integration RuntimesGET /scan/api/integrationruntimesLists on-premises SHIR agents for scanning local DBs.
137Get Self-Hosted IR StatusGET /scan/api/integrationruntimes/{irName}/statusInspects health and CPU/RAM load of SHIR agent.
138Get Self-Hosted IR KeyPOST /scan/api/integrationruntimes/{irName}/listauthkeysGenerates registration key for local SHIR installation.
139Regenerate Self-Hosted IR KeyPOST /scan/api/integrationruntimes/{irName}/regenerateauthkeyRotates security registration key for SHIR agent.
140Delete Self-Hosted IRDELETE /scan/api/integrationruntimes/{irName}Removes on-premises SHIR connector.
141Get Scan ScheduleGET /scan/api/datasources/{ds}/scans/{scanName}/triggers/defaultInspects recurring cron scanning schedule.
142Set Scan Schedule CronPUT /scan/api/datasources/{ds}/scans/{scanName}/triggers/defaultSchedules weekly/monthly automated scan cron.
143Disable Scan ScheduleDELETE /scan/api/datasources/{ds}/scans/{scanName}/triggers/defaultDeactivates automated scan schedule.
144Get Scan Execution Diagnostics LogGET /scan/api/datasources/{ds}/scans/{scanName}/runs/{runId}/errorPulls error log trace for failed scan run.
145Get Custom Classification Rule RegExGET /scan/api/classificationrules/{ruleName}/patternsInspects regular expression pattern for classifier.
146Get Classification Rule DictionaryGET /scan/api/classificationrules/{ruleName}/dictionaryInspects match dictionary terms for classifier.
147Get Sensitivity Label Policy TargetGET /policy/api/sensitivitylabels/policiesInspects MIP sensitivity label auto-apply policies.
148Create Sensitivity Label RulePOST /policy/api/sensitivitylabels/rulesProvisions rule auto-labeling assets matching classifications.
149Get Policy Enforcement PointGET /policy/api/enforcement/statusVerifies policy enforcement state across target storage.
150Create Data Governance RolePOST /account/api/rolesProvisions custom role in Purview access control.
151Get User Assigned RolesGET /account/api/users/{userId}/rolesLists Purview roles assigned to specific Azure AD principal.
152Grant User Collection AdminPOST /account/api/collections/{col}/roleAssignmentsGrants Collection Admin role to Azure AD principal.
153Grant User Data CuratorPOST /account/api/collections/{col}/roleAssignmentsGrants Data Curator role to Azure AD principal.
154Grant User Data ReaderPOST /account/api/collections/{col}/roleAssignmentsGrants Data Reader role to Azure AD principal.
155Grant User Workflow AdminPOST /account/api/collections/{col}/roleAssignmentsGrants Workflow Admin role to Azure AD principal.
156List Data WorkflowsGET /workflow/api/workflowsLists automated data governance workflows.
157Create Data WorkflowPOST /workflow/api/workflowsProvisions approval workflow for term creation/asset edits.
158Update Data WorkflowPUT /workflow/api/workflows/{workflowId}Modifies BPMN approval workflow steps.
159Delete Data WorkflowDELETE /workflow/api/workflows/{workflowId}Removes governance workflow.
160Get Workflow Run HistoryGET /workflow/api/workflowrunsInspects pending and completed workflow executions.
161Approve Workflow RequestPOST /workflow/api/userrequests/{requestId}/approveApproves pending governance approval ticket.
162Reject Workflow RequestPOST /workflow/api/userrequests/{requestId}/rejectRejects pending governance approval ticket.
163Get User Inbox TasksGET /workflow/api/userrequests/pendingLists active pending approval tasks assigned to current user.
164Get Metastore Sync LogsGET /catalog/api/metastore/sync/logsPulls execution logs for Hive Metastore / Unity sync.
165Get Data Map Search AutocompletePOST /catalog/api/search/autocompleteProvides real-time search term completion suggestions.
166Get Data Map Search SuggestPOST /catalog/api/search/suggestProvides fuzzy asset search suggestions.
167Get Asset Relationships ListGET /catalog/api/atlas/v2/entity/guid/{guid}/relationshipsLists all connected edge relationships for asset.
168Get Asset Classification HistoryGET /catalog/api/atlas/v2/entity/guid/{guid}/classificationHistoryAudit trail of applied/removed classifications.
169Get Data Map Schema DriftGET /insights/api/schema/driftIdentifies tables where physical schema drifted from catalog.
170Get Unclassified Assets ReportGET /insights/api/assets/unclassifiedLists assets lacking applied classifications.
171Get Uncurated Assets ReportGET /insights/api/assets/uncuratedLists assets lacking business term or owner assignment.
172Get Stale Assets ReportGET /insights/api/assets/staleLists assets untouched by scans in > 90 days.
173Set Account Storage FirewallSet-AzPurviewAccount -Name 'org-purview' -PublicNetworkAccess DisabledBlocks public internet traffic to Purview endpoint.
174Get Account Private Endpoint ConnectionsGet-AzPurviewPrivateEndpointConnection -AccountName 'org-purview'Lists active private endpoint connections.
175Delete Private Endpoint ConnectionRemove-AzPurviewPrivateEndpointConnection -AccountName 'org-purview' -Name 'pe-1'Deletes private endpoint connection.
176Get Account Managed IdentityGet-AzPurviewAccount -Name 'org-purview' | Select-Object IdentityInspects System-Assigned Managed Identity Object ID.
177Grant Managed Identity Key Vault AccessSet-AzKeyVaultAccessPolicy -VaultName 'kv' -ObjectId '{purviewMsiId}' -PermissionsToSecrets Get,ListGrants Purview MSI access to Key Vault.
178Grant Managed Identity Storage AccessNew-AzRoleAssignment -ObjectId '{purviewMsiId}' -RoleDefinitionName 'Storage Blob Data Reader' -Scope '/subscriptions/123/...'Grants Purview MSI storage read rights.
179Grant Managed Identity SQL AccessCREATE USER [org-purview] FROM EXTERNAL PROVIDER; ALTER ROLE db_datareader ADD MEMBER [org-purview];Grants Purview MSI SQL Server read rights.
180Get Notification Event Stream ConfigGET /account/api/notification/configInspects Kafka/Event Hub streaming hooks for data map mutations.
181Set Notification Event Stream ConfigPUT /account/api/notification/configConfigures streaming topic for Purview atlas mutation events.
182Test Kafka Notification ProducerPOST /account/api/notification/testSends test payload to Kafka/Event Hub topic.
183Get Catalog Audit TrailGET /account/api/audit/logsPulls administrative modification audit logs.
184Get Metastore Entity CountsGET /catalog/api/atlas/v2/types/typedefs/summaryOutputs entity counts categorized by type.
185Export Catalog Lineage GraphGET /catalog/api/atlas/v2/lineage/exportExports lineage DAG to GraphML/JSON format.
186Get Glossary Term Access ControlGET /catalog/api/glossary/terms/{guid}/accessControlInspects ABAC access policies applied to term.
187Set Glossary Term Access ControlPUT /catalog/api/glossary/terms/{guid}/accessControlRestricts term edit/view permissions to specific roles.
188Get Purview API VersionGET /account/api/versionOutputs Purview REST API build version string.
189Check Data Map PingGET /account/api/pingHealth check endpoint returning 200 OK.
190Get Data Map Search FiltersPOST /catalog/api/search/filtersInspects available search facets and filters.
191Get Data Map Asset OwnersGET /catalog/api/atlas/v2/entity/guid/{guid}/ownersLists assigned business and technical owners.
192Set Data Map Asset OwnerPOST /catalog/api/atlas/v2/entity/guid/{guid}/ownersAssigns business or technical owner to asset.
193Remove Data Map Asset OwnerDELETE /catalog/api/atlas/v2/entity/guid/{guid}/ownersRemoves assigned owner from asset.
194Get Data Map Asset ExpertsGET /catalog/api/atlas/v2/entity/guid/{guid}/expertsLists assigned subject matter experts.
195Set Data Map Asset ExpertPOST /catalog/api/atlas/v2/entity/guid/{guid}/expertsAssigns subject matter expert to asset.
196Remove Data Map Asset ExpertDELETE /catalog/api/atlas/v2/entity/guid/{guid}/expertsRemoves subject matter expert from asset.
197Get Asset Schema ColumnsGET /catalog/api/atlas/v2/entity/guid/{guid}/schemaRetrieves column schema definitions for table asset.
198Get Column Classification DetailsGET /catalog/api/atlas/v2/entity/guid/{guid}/classificationsLists classifications applied to column.
199Add Column ClassificationPOST /catalog/api/atlas/v2/entity/guid/{guid}/classificationsApplies classification tag to column.
200Remove Column ClassificationDELETE /catalog/api/atlas/v2/entity/guid/{guid}/classifications/{name}Removes classification tag from column.

Collibra Governance Gov

Technical Architecture & Overview

Collibra Governance is an enterprise Data Intelligence Cloud platform that combines data governance, data catalog, data lineage, and data quality into a unified metadata repository. It operates on an operating model composed of Communities, Domains, Asset Types, and Attributes, providing an enterprise-wide system of record for data assets, policies, and business concepts.

Primary Use Cases: Enterprise data governance stewardship, data privacy and compliance (GDPR, CCPA, BCBS 239), business glossary standardization, automated pipeline impact analysis, workflow-driven data access approvals, and data quality rule enforcement.

Core Components: Collibra DGC (Operating Model & Governance Engine), Collibra Catalog (metadata ingestion & automated classification), Collibra Lineage (automated harvest of SQL, ETL, BI scripts), Collibra Edge (on-premises secure agent), and BPMN Workflow Engine.

Exhaustive 200-command administrative, diagnostic, and operational matrix for Collibra Governance.

#Operation / ActionAPI Endpoint / SyntaxDescription
1Get All AssetsGET /rest/2.0/assetsRetrieves paginated catalog assets.
2Create AssetPOST /rest/2.0/assetsProvisions a new catalog object.
3Update AssetPATCH /rest/2.0/assets/{id}Modifies asset properties.
4Delete AssetDELETE /rest/2.0/assets/{id}Removes an asset from the catalog.
5Find Asset by IDGET /rest/2.0/assets/{id}Fetches a single asset's details.
6List DomainsGET /rest/2.0/domainsRetrieves governance domains.
7Create DomainPOST /rest/2.0/domainsCreates a logical asset domain.
8List CommunitiesGET /rest/2.0/communitiesFetches organizational communities.
9Create CommunityPOST /rest/2.0/communitiesEstablishes a governance community.
10Get Asset LineageGET /rest/2.0/lineage/{id}/upstreamInspects data lineage upstream.
11Create RelationPOST /rest/2.0/relationsLinks two catalog assets together.
12List RelationsGET /rest/2.0/relationsRetrieves asset dependencies.
13Assign ResponsibilityPOST /rest/2.0/responsibilitiesDelegates data stewardship.
14Get Data Quality RulesGET /rest/2.0/dataQualityRulesFetches DQ validation logic.
15Create Data Quality RulePOST /rest/2.0/dataQualityRulesAdds a metric validation rule.
16List UsersGET /rest/2.0/usersRetrieves platform user accounts.
17Create UserPOST /rest/2.0/usersProvisions a new platform user.
18List GroupsGET /rest/2.0/userGroupsRetrieves user permission groups.
19Assign User to GroupPOST /rest/2.0/userGroups/{id}/usersAdds user to security group.
20Export Workflow DefinitionGET /rest/2.0/workflows/processesFetches BPMN governance workflows.
21Deploy WorkflowPOST /rest/2.0/workflows/processesUploads a new approval workflow.
22Get Workflow TasksGET /rest/2.0/workflowTasksLists pending approval tasks.
23Complete Workflow TaskPOST /rest/2.0/workflowTasks/{id}/completeApproves/rejects a governance ticket.
24Search Global IndexGET /rest/2.0/searchExecutes enterprise catalog search.
25Get Asset AttributesGET /rest/2.0/attributes?assetId={id}Retrieves metadata attributes.
26Create AttributePOST /rest/2.0/attributesAdds custom metadata fields.
27Get Asset TypesGET /rest/2.0/assetTypesLists available asset type schemas.
28Create Asset TypePOST /rest/2.0/assetTypesProvisions a custom asset type.
29Get System HealthGET /rest/2.0/system/statusVerifies backend engine status.
30Purge TrashPOST /rest/2.0/trash/emptyPermanently deletes archived objects.
31Get Domain DetailsGET /rest/2.0/domains/{id}Retrieves properties of a specific domain.
32Update DomainPATCH /rest/2.0/domains/{id}Modifies domain attributes or parent community.
33Delete DomainDELETE /rest/2.0/domains/{id}Removes a domain and optionally contents.
34Get Community DetailsGET /rest/2.0/communities/{id}Retrieves details of a specific community.
35Update CommunityPATCH /rest/2.0/communities/{id}Modifies community properties.
36Delete CommunityDELETE /rest/2.0/communities/{id}Removes a community node.
37Get Downstream LineageGET /rest/2.0/lineage/{id}/downstreamInspects downstream impact analysis.
38Get Full Lineage GraphGET /rest/2.0/lineage/{id}/graphExtracts complete upstream and downstream DAG.
39Get Relation TypeGET /rest/2.0/relationTypes/{id}Inspects definition of a relation type.
40Create Relation TypePOST /rest/2.0/relationTypesDefines new relationship schema between asset types.
41Delete RelationDELETE /rest/2.0/relations/{id}Removes relationship link between assets.
42List ResponsibilitiesGET /rest/2.0/responsibilitiesLists data stewardship assignments.
43Delete ResponsibilityDELETE /rest/2.0/responsibilities/{id}Revokes stewardship assignment from user.
44Get RoleGET /rest/2.0/roles/{id}Retrieves security role definition.
45List RolesGET /rest/2.0/rolesLists all system and custom governance roles.
46Create RolePOST /rest/2.0/rolesProvisions a new custom governance role.
47Get Attribute TypeGET /rest/2.0/attributeTypes/{id}Inspects schema of attribute type definition.
48Create Attribute TypePOST /rest/2.0/attributeTypesProvisions custom attribute definition.
49Update AttributePATCH /rest/2.0/attributes/{id}Modifies value of an existing asset attribute.
50Delete AttributeDELETE /rest/2.0/attributes/{id}Removes an attribute from an asset.
51Get Complex RelationGET /rest/2.0/complexRelations/{id}Retrieves multi-way relationship object.
52Create Complex RelationPOST /rest/2.0/complexRelationsProvisions N-ary relation among multiple assets.
53Delete Complex RelationDELETE /rest/2.0/complexRelations/{id}Removes multi-way relationship object.
54Get DQ Metric ResultsGET /rest/2.0/dataQualityResultsFetches history of DQ execution metrics.
55Add DQ Metric ResultPOST /rest/2.0/dataQualityResultsPosts new DQ test execution score.
56Get User ProfileGET /rest/2.0/users/{id}Fetches detailed account info for a user.
57Update User AccountPATCH /rest/2.0/users/{id}Updates email, roles, or status of a user.
58Disable User AccountPOST /rest/2.0/users/{id}/disableDeactivates a user login.
59Enable User AccountPOST /rest/2.0/users/{id}/enableReactivates a user login.
60Get User GroupGET /rest/2.0/userGroups/{id}Retrieves user group properties.
61Remove User from GroupDELETE /rest/2.0/userGroups/{id}/users/{userId}Removes a user from permission group.
62Get Workflow InstanceGET /rest/2.0/workflows/instances/{id}Inspects running BPMN workflow instance.
63Start Workflow InstancePOST /rest/2.0/workflows/processes/{id}/startTriggers execution of governance workflow.
64Cancel Workflow InstancePOST /rest/2.0/workflows/instances/{id}/cancelAborts a running workflow process.
65List Workflow Tasks for UserGET /rest/2.0/workflowTasks?userId={userId}Lists pending inbox tasks for specific user.
66Reassign Workflow TaskPOST /rest/2.0/workflowTasks/{id}/reassignDelegates approval task to another user.
67Get CommentGET /rest/2.0/comments/{id}Retrieves specific asset discussion comment.
68Add Comment to AssetPOST /rest/2.0/commentsPosts a collaboration note on catalog asset.
69Delete CommentDELETE /rest/2.0/comments/{id}Removes discussion comment.
70Get AttachmentGET /rest/2.0/attachments/{id}Retrieves file attachment metadata.
71Upload AttachmentPOST /rest/2.0/attachmentsAttaches document/spec file to catalog asset.
72Delete AttachmentDELETE /rest/2.0/attachments/{id}Removes attached file from asset.
73Get TagGET /rest/2.0/tags/{id}Retrieves specific catalog tag.
74Add Tag to AssetPOST /rest/2.0/tags/assignApplies descriptive tag to asset.
75Remove Tag from AssetPOST /rest/2.0/tags/unassignRemoves descriptive tag from asset.
76Get View CountGET /rest/2.0/navigation/mostViewedRetrieves popularity metrics for catalog assets.
77Export Catalog Assets CSVPOST /rest/2.0/outputModule/export/csvExports search or domain assets to CSV.
78Export Catalog Assets ExcelPOST /rest/2.0/outputModule/export/excelExports asset catalog to Excel file.
79Import Data via JobPOST /rest/2.0/jobs/importBulk imports catalog entities via JSON/CSV.
80Get Job StatusGET /rest/2.0/jobs/{id}Monitors background import/export batch job.
81Cancel JobPOST /rest/2.0/jobs/{id}/cancelStops background processing batch job.
82Get System InfoGET /rest/2.0/system/infoInspects version, license, and cluster build.
83Get License DetailsGET /rest/2.0/system/licenseInspects active Collibra license tier and quotas.
84Get Audit LogsGET /rest/2.0/auditLogsFetches security and administrative audit event trail.
85Reindex Search IndexPOST /rest/2.0/search/reindexTriggers full catalog search index rebuild.
86Get Edge Site StatusGET /rest/2.0/edge/sitesMonitors status of Collibra Edge connectors.
87Create Edge Integration JobPOST /rest/2.0/edge/jobsSchedules ingestion job via Edge site.
88Get Ingestion CapabilityGET /rest/2.0/edge/capabilitiesInspects supported databases on Edge sites.
89List Governance RulesGET /rest/2.0/rulesLists business policies and validation logic.
90Create Governance RulePOST /rest/2.0/rulesProvisions new policy validation rule.
91Delete Governance RuleDELETE /rest/2.0/rules/{id}Removes governance policy rule.
92Get Custom IconGET /rest/2.0/icons/{id}Retrieves custom UI icon definition.
93Upload Custom IconPOST /rest/2.0/iconsUploads custom asset icon graphics.
94Get Mail ConfigurationGET /rest/2.0/mail/statusInspects SMTP email notification server.
95Send Test MailPOST /rest/2.0/mail/testSends test email to verify SMTP gateway.
96Get LDAP ConfigGET /rest/2.0/ldap/configurationInspects enterprise directory integration.
97Sync LDAP UsersPOST /rest/2.0/ldap/syncExecutes directory user synchronization.
98Get SSO SAML MetadataGET /rest/2.0/sso/saml/metadataExports SAML service provider metadata.
99Purge Asset PermanentlyDELETE /rest/2.0/assets/{id}/purgeBypasses trash and purges asset.
100Restore Asset from TrashPOST /rest/2.0/trash/restoreRestores archived asset from trash bin.
101Purge Asset PermanentlyDELETE /rest/2.0/assets/{id}/purgeBypasses trash and purges asset.
102Restore Asset from TrashPOST /rest/2.0/trash/restoreRestores archived asset from trash bin.
103Get All Communities PaginatedGET /rest/2.0/communities?limit=100&offset=0Paginates through organizational communities.
104Get Sub-CommunitiesGET /rest/2.0/communities?parentId={id}Lists nested sub-communities under parent.
105Get Community DomainsGET /rest/2.0/domains?communityId={id}Lists governance domains within community.
106Get Domain Assets PaginatedGET /rest/2.0/assets?domainId={id}&limit=100Paginates assets contained in domain.
107Change Asset DomainPOST /rest/2.0/assets/{id}/domain/{targetDomainId}Moves asset to different governance domain.
108Change Asset NamePATCH /rest/2.0/assets/{id}Updates primary display name of asset.
109Change Asset TypePOST /rest/2.0/assets/{id}/type/{targetTypeId}Converts asset to different asset type.
110Get Asset StatusGET /rest/2.0/assets/{id}/statusInspects lifecycle status (Candidate, Approved, Obsolete).
111Set Asset StatusPOST /rest/2.0/assets/{id}/status/{statusId}Updates lifecycle status of asset.
112Get Asset Relations By TypeGET /rest/2.0/relations?sourceId={id}&relationTypeId={relTypeId}Lists relations matching specific type.
113Delete Relations BatchDELETE /rest/2.0/relations/bulkBatch deletes relationship links.
114Get Complex Relation TypeGET /rest/2.0/complexRelationTypes/{id}Inspects definition of complex relation type.
115List Complex Relation TypesGET /rest/2.0/complexRelationTypesLists all complex relation schemas.
116Create Complex Relation TypePOST /rest/2.0/complexRelationTypesProvisions new multi-way relation schema.
117Delete Complex Relation TypeDELETE /rest/2.0/complexRelationTypes/{id}Deletes complex relation schema.
118Get Attribute Types By GroupGET /rest/2.0/attributeTypes?groupId={id}Lists attribute types belonging to group.
119List Attribute Type GroupsGET /rest/2.0/attributeTypeGroupsLists attribute type organization groups.
120Create Attribute Type GroupPOST /rest/2.0/attributeTypeGroupsProvisions new attribute organization group.
121Get Asset Type AssignmentGET /rest/2.0/assignments?assetTypeId={id}Inspects allowed attributes/relations for asset type.
122Create Asset Type AssignmentPOST /rest/2.0/assignmentsBinds attribute/relation types to an asset type.
123Update Asset Type AssignmentPATCH /rest/2.0/assignments/{id}Modifies allowed attribute cardinality/rules.
124Delete Asset Type AssignmentDELETE /rest/2.0/assignments/{id}Unbinds attribute/relation types from asset type.
125Get Scope DefinitionGET /rest/2.0/scopes/{id}Inspects scope governing assignment rules.
126List ScopesGET /rest/2.0/scopesLists global and domain scopes.
127Create ScopePOST /rest/2.0/scopesProvisions governance assignment scope.
128Get Stewardship Role AssignmentsGET /rest/2.0/responsibilities?roleId={roleId}Lists all stewards assigned to specific role.
129Add Responsibility BatchPOST /rest/2.0/responsibilities/bulkBatch assigns stewardship roles across assets.
130Remove Responsibility BatchDELETE /rest/2.0/responsibilities/bulkBatch revokes stewardship assignments.
131Get User Permissions SummaryGET /rest/2.0/users/{id}/permissionsCalculates global and domain permissions for user.
132Get Group Permissions SummaryGET /rest/2.0/userGroups/{id}/permissionsCalculates permissions for security group.
133Grant Global PermissionPOST /rest/2.0/permissionsGrants global administrative permission to group/role.
134Revoke Global PermissionDELETE /rest/2.0/permissions/{id}Revokes global administrative permission.
135Get Workflow Process DefinitionGET /rest/2.0/workflows/processes/{id}Retrieves BPMN XML definition of workflow.
136Update Workflow Process DefinitionPUT /rest/2.0/workflows/processes/{id}Updates BPMN XML code of workflow.
137Delete Workflow Process DefinitionDELETE /rest/2.0/workflows/processes/{id}Deletes governance workflow definition.
138Get Workflow Task Form DataGET /rest/2.0/workflowTasks/{id}/formPropertiesFetches required user input fields for task.
139Submit Task Form DataPOST /rest/2.0/workflowTasks/{id}/completedSubmits completed form data to advance workflow.
140Get Workflow VariableGET /rest/2.0/workflows/instances/{id}/variables/{name}Inspects process variable in workflow instance.
141Set Workflow VariablePUT /rest/2.0/workflows/instances/{id}/variables/{name}Modifies process variable in workflow instance.
142Get Data Quality Rule DefinitionGET /rest/2.0/dataQualityRules/{id}Inspects pass/fail logic of DQ rule.
143Update Data Quality RulePATCH /rest/2.0/dataQualityRules/{id}Modifies threshold or metric logic of DQ rule.
144Delete Data Quality RuleDELETE /rest/2.0/dataQualityRules/{id}Deletes DQ rule definition.
145Get DQ Metric AggregationsGET /rest/2.0/dataQualityResults/aggregationCalculates average DQ pass rates by domain.
146Get Lineage Harvesting Execution LogsGET /rest/2.0/lineage/harvests/{id}/logsPulls execution log for lineage extraction job.
147Trigger Lineage Harvesting JobPOST /rest/2.0/lineage/harvestsTriggers automated lineage harvesting execution.
148Cancel Lineage Harvesting JobPOST /rest/2.0/lineage/harvests/{id}/cancelAborts running lineage harvesting job.
149Get Edge Connector Site DetailGET /rest/2.0/edge/sites/{id}Inspects health, host, and version of Edge site.
150Upgrade Edge Connector SitePOST /rest/2.0/edge/sites/{id}/upgradeTriggers remote software upgrade of Edge agent.
151Get Edge Connection DetailsGET /rest/2.0/edge/connections/{id}Inspects JDBC connection parameters on Edge site.
152Create Edge ConnectionPOST /rest/2.0/edge/connectionsRegisters new JDBC data source on Edge site.
153Test Edge ConnectionPOST /rest/2.0/edge/connections/{id}/testTests JDBC connectivity from Edge site to database.
154Delete Edge ConnectionDELETE /rest/2.0/edge/connections/{id}Removes JDBC data source from Edge site.
155Get Navigation Taxonomy TreeGET /rest/2.0/navigation/treeFetches hierarchical navigation tree.
156Get Most Frequent Search QueriesGET /rest/2.0/search/topQueriesPulls top search keywords typed by users.
157Get Asset View StatisticsGET /rest/2.0/assets/{id}/statisticsInspects page views and unique visitors for asset.
158Get Comment ThreadGET /rest/2.0/comments?assetId={id}Retrieves threaded comments on catalog asset.
159Update CommentPATCH /rest/2.0/comments/{id}Modifies text of posted comment.
160Get Asset RatingsGET /rest/2.0/ratings?assetId={id}Calculates average star rating for asset.
161Add Asset RatingPOST /rest/2.0/ratingsSubmits 1-5 star rating and review for asset.
162Delete Asset RatingDELETE /rest/2.0/ratings/{id}Removes star rating from asset.
163Get Asset FavoritesGET /rest/2.0/favorites?userId={id}Lists assets bookmarked by user.
164Add Asset to FavoritesPOST /rest/2.0/favoritesBookmarks asset for current user.
165Remove Asset from FavoritesDELETE /rest/2.0/favorites/{id}Removes bookmarked asset for current user.
166Get Asset SubscriptionGET /rest/2.0/subscriptions?assetId={id}Lists users subscribed to notification updates on asset.
167Subscribe to AssetPOST /rest/2.0/subscriptionsSubscribes user to email notifications for asset modifications.
168Unsubscribe from AssetDELETE /rest/2.0/subscriptions/{id}Unsubscribes user from asset notifications.
169Get Active Session CountGET /rest/2.0/system/sessionsLists active HTTP user sessions.
170Terminate User SessionDELETE /rest/2.0/system/sessions/{sessionId}Force-logs out active user session.
171Get Cache ConfigurationGET /rest/2.0/system/cacheInspects status of in-memory application caches.
172Flush Specific CachePOST /rest/2.0/system/cache/{cacheName}/flushPurges specific application cache.
173Flush All System CachesPOST /rest/2.0/system/cache/flushAllPurges all in-memory application caches.
174Get Backup Export FileGET /rest/2.0/system/backupDownloads platform configuration backup ZIP archive.
175Restore Platform BackupPOST /rest/2.0/system/restoreRestores platform configuration from ZIP archive.
176Get Maintenance Mode StatusGET /rest/2.0/system/maintenanceInspects whether maintenance mode is active.
177Enable Maintenance ModePOST /rest/2.0/system/maintenance/enablePuts Collibra in read-only maintenance mode.
178Disable Maintenance ModePOST /rest/2.0/system/maintenance/disableRestores normal read-write operation.
179Get SAML Identity Provider ConfigGET /rest/2.0/sso/saml/configInspects SAML IdP configuration settings.
180Update SAML Identity Provider ConfigPUT /rest/2.0/sso/saml/configUpdates SAML IdP certificate or SSO URL.
181Get LDAP Synchronization StatusGET /rest/2.0/ldap/statusInspects background LDAP user directory sync.
182Cancel LDAP SynchronizationPOST /rest/2.0/ldap/cancelAborts running LDAP user directory sync.
183Get Mail Queue StatusGET /rest/2.0/mail/queueInspects pending email notification queue.
184Purge Mail QueueDELETE /rest/2.0/mail/queueClears pending email notification queue.
185Get API Rate Limit QuotasGET /rest/2.0/system/ratelimitsInspects REST API rate limit counters per IP/user.
186Get System Thread DumpGET /rest/2.0/system/threaddumpDownloads JVM thread stack trace dump for diagnostics.
187Get System Memory MetricsGET /rest/2.0/system/memoryInspects JVM heap, non-heap, and GC stats.
188Get Database Migration StatusGET /rest/2.0/system/db/migrationInspects database Liquibase schema migration status.
189Check Collibra API PingGET /rest/2.0/system/pingHealth check endpoint returning 200 OK.
190Get Asset Attribute HistoryGET /rest/2.0/attributes/{id}/historyAudit log of attribute value modifications.
191Get Asset Relation HistoryGET /rest/2.0/relations/{id}/historyAudit log of relation link modifications.
192Get Asset Type DetailsGET /rest/2.0/assetTypes/{id}Inspects structural schema for asset type.
193Update Asset TypePATCH /rest/2.0/assetTypes/{id}Modifies asset type name, icon, or description.
194Delete Asset TypeDELETE /rest/2.0/assetTypes/{id}Removes asset type schema definition.
195Get Community HierarchyGET /rest/2.0/communities/hierarchyFetches full community hierarchy tree.
196Get Domain TypesGET /rest/2.0/domainTypesLists all domain type schemas.
197Get Domain Type DetailsGET /rest/2.0/domainTypes/{id}Inspects structural schema for domain type.
198Create Domain TypePOST /rest/2.0/domainTypesProvisions custom domain type schema.
199Delete Domain TypeDELETE /rest/2.0/domainTypes/{id}Deletes custom domain type schema.
200Get Role DetailsGET /rest/2.0/roles/{id}Inspects permissions assigned to role.

Alation Catalog Sync Gov

Technical Architecture & Overview

Alation Catalog Sync provides automated metadata extraction, data search and discovery, business glossaries, and behavioral analytics. It interfaces with relational databases, cloud warehouses, BI tools, and data lakehouses using the Open Connector Framework (OCF) and native REST APIs.

Primary Use Cases: Self-service data discovery, SQL query endorsement and sharing via Alation Compose, data stewardship scoring, automated data dictionary curation, and BI dashboard lineage tracking.

Core Components: Alation Server (Django/Postgres metadata store), Open Connector Framework (OCF execution engine), Compose (collaborative SQL editor), and Lexicon (automated term extraction).

Exhaustive 200-command administrative, diagnostic, and operational matrix for Alation Catalog Sync.

#Operation / ActionAPI Endpoint / CommandDescription
1Get Catalog SourcesGET /integration/v1/sources/Lists registered database sources.
2Trigger Metadata ExtractionPOST /integration/v1/source/{id}/extract/Pulls schema schemas from source.
3Get Custom FieldsGET /integration/v1/fields/Retrieves custom metadata attributes.
4Set Field ValuePOST /integration/v1/field_value/Updates custom asset property.
5Get Table MetadataGET /integration/v1/table/{table_id}/Retrieves table-level properties.
6Get Column MetadataGET /integration/v1/column/{column_id}/Fetches column-level descriptions.
7Create ArticlePOST /integration/v1/article/Publishes a documentation wiki page.
8List ArticlesGET /integration/v1/article/Retrieves all documentation pages.
9Upload Lineage GraphPOST /integration/v1/lineage/Inbound upstream/downstream flow.
10Get Lineage GraphGET /integration/v1/lineage/table/{id}/Retrieves table data provenance.
11List UsersGET /integration/v1/user/Fetches registered catalog users.
12Add Endorsed QueryPOST /integration/v1/query/Publishes a certified SQL query.
13Search CatalogGET /api/v1/search/?q={query}Executes metadata search query.
14Get BI ReportsGET /integration/v1/bi_reports/Retrieves connected Tableau/PowerBI items.
15Push BI LineagePOST /integration/v1/bi_lineage/Syncs dashboard data dependencies.
16Get Glossary TermsGET /integration/v1/glossary/terms/Fetches enterprise dictionary.
17Create Glossary TermPOST /integration/v1/glossary/terms/Adds a business data term.
18Link Term to ColumnPOST /integration/v1/glossary/mapping/Associates term with physical column.
19Get Stewardship MetricsGET /integration/v1/stewardship/Evaluates catalog completeness score.
20Trigger OCF Connector SyncPOST /api/v1/datasource/{id}/run_job/Executes Open Connector Framework job.
21Get Job StatusGET /api/v1/job/{job_id}/Checks background sync task status.
22Export Schema DumpGET /integration/v1/export/schema/Downloads metadata backup payload.
23Import Schema MetadataPOST /integration/v1/import/schema/Bulk ingests metadata structures.
24Get User GroupsGET /integration/v1/group/Lists permission roles and groups.
25Assign Group PermissionsPOST /integration/v1/group_permissions/Grants dataset access rights.
26Get Deleted AssetsGET /integration/v1/trash/Inspects catalog recycle bin.
27Restore Deleted AssetPOST /integration/v1/trash/{id}/restore/Recovers archived catalog objects.
28Get System LogsGET /api/v1/admin/logs/Pulls container and daemon logs.
29Test API TokenGET /integration/v1/auth/test/Validates authentication credentials.
30Restart Alation Workersudo systemctl restart alation-workerRestarts background processing worker.
31Get Data Source DetailsGET /integration/v1/datasource/{id}/Fetches configuration of specific data source.
32Create Data SourcePOST /integration/v1/datasource/Registers a new relational or cloud database.
33Update Data SourcePUT /integration/v1/datasource/{id}/Modifies connection credentials or host parameters.
34Delete Data SourceDELETE /integration/v1/datasource/{id}/Removes data source registration from catalog.
35Get Schema ListGET /integration/v1/schema/?ds_id={id}Lists all database schemas within a data source.
36Get Schema DetailsGET /integration/v1/schema/{id}/Retrieves metadata properties of a schema.
37Update Schema MetadataPOST /integration/v1/schema/{id}/Updates schema description or stewardship tags.
38Update Table MetadataPOST /integration/v1/table/{id}/Updates table title, description, or custom fields.
39Delete Table MetadataDELETE /integration/v1/table/{id}/Removes table entity from Alation catalog.
40Update Column MetadataPOST /integration/v1/column/{id}/Modifies column description or security tags.
41Bulk Update ColumnsPOST /integration/v1/column/bulk_update/Batch updates descriptions for multiple columns.
42Get Query DetailsGET /integration/v1/query/{id}/Fetches SQL text and metadata of saved query.
43Update QueryPUT /integration/v1/query/{id}/Modifies SQL code or endorsement status.
44Delete QueryDELETE /integration/v1/query/{id}/Removes saved query from Compose catalog.
45Run Query in ComposePOST /api/v1/compose/execute/Executes query against target database engine.
46Get Query Execution ResultGET /api/v1/compose/result/{id}/Fetches output dataset of executed query.
47Get Article DetailsGET /integration/v1/article/{id}/Fetches title and content of wiki article.
48Update ArticlePUT /integration/v1/article/{id}/Modifies documentation article content.
49Delete ArticleDELETE /integration/v1/article/{id}/Removes wiki documentation page.
50Get Custom Field DefinitionGET /integration/v1/fields/{id}/Inspects schema of custom field attribute.
51Create Custom FieldPOST /integration/v1/fields/Provisions custom metadata field for catalog objects.
52Delete Custom FieldDELETE /integration/v1/fields/{id}/Removes custom metadata field definition.
53Get User DetailsGET /integration/v1/user/{id}/Fetches user profile, email, and group membership.
54Create User AccountPOST /integration/v1/user/Provisions user account in Alation.
55Update User AccountPUT /integration/v1/user/{id}/Modifies user role, name, or active status.
56Deactivate User AccountDELETE /integration/v1/user/{id}/Deactivates user login in Alation.
57Get User Group DetailsGET /integration/v1/group/{id}/Retrieves group membership and assigned roles.
58Create User GroupPOST /integration/v1/group/Provisions security group in Alation.
59Add User to GroupPOST /integration/v1/group/{id}/add_user/Adds user account to security group.
60Remove User from GroupPOST /integration/v1/group/{id}/remove_user/Removes user account from security group.
61Get Trust FlagGET /integration/v1/trust_flag/Lists endorsement and deprecation flags.
62Create Trust FlagPOST /integration/v1/trust_flag/Applies Warning or Endorsement tag on asset.
63Delete Trust FlagDELETE /integration/v1/trust_flag/{id}/Removes Warning or Endorsement tag from asset.
64Get Glossary Term DetailsGET /integration/v1/glossary/terms/{id}/Retrieves definition and relationships of term.
65Update Glossary TermPUT /integration/v1/glossary/terms/{id}/Modifies business term definition or acronym.
66Delete Glossary TermDELETE /integration/v1/glossary/terms/{id}/Removes term from business glossary.
67Get Data Dict ExportGET /integration/v1/data_dictionary/export/Exports data dictionary to CSV/Excel format.
68Import Data Dict CSVPOST /integration/v1/data_dictionary/import/Bulk imports column descriptions from CSV.
69Get Profiling ResultsGET /integration/v1/profiling/table/{id}/Fetches row counts, null % and distinct counts.
70Trigger Data ProfilingPOST /integration/v1/source/{id}/profile/Runs automated data profiling job.
71Get Sampling ConfigGET /integration/v1/source/{id}/sample_config/Inspects sample query limits for profiling.
72Set Sampling ConfigPUT /integration/v1/source/{id}/sample_config/Modifies sample size limits for profiling.
73Get Lineage NodeGET /integration/v1/lineage/node/{id}/Retrieves specific node details in lineage graph.
74Delete Lineage LinkDELETE /integration/v1/lineage/link/{id}/Removes automated or manual lineage connection.
75Get BI Server DetailsGET /integration/v1/bi_server/{id}/Inspects connected Tableau/PowerBI server.
76Register BI ServerPOST /integration/v1/bi_server/Registers BI server for dashboard cataloging.
77Trigger BI Server SyncPOST /integration/v1/bi_server/{id}/sync/Initiates metadata extraction from BI server.
78Get Policy ListGET /integration/v1/policy/Lists access policies and data privacy rules.
79Create Data Privacy PolicyPOST /integration/v1/policy/Provisions data masking or restricted access rule.
80Delete Data PolicyDELETE /integration/v1/policy/{id}/Removes data governance policy.
81Get API Token ListGET /integration/v1/auth/tokens/Lists active API refresh tokens.
82Create API TokenPOST /integration/v1/auth/create_token/Generates new API authorization token.
83Revoke API TokenPOST /integration/v1/auth/revoke_token/Revokes API token credentials.
84Get Catalog Usage MetricsGET /api/v1/analytics/usage/Pulls active user counts and top viewed tables.
85Get Search PopularityGET /api/v1/analytics/search_top/Inspects most frequent user search terms.
86Reindex Search IndexPOST /api/v1/admin/reindex/Triggers search engine reindexing task.
87Get Backup ScheduleGET /api/v1/admin/backup/config/Inspects automated database backup schedule.
88Trigger Manual BackupPOST /api/v1/admin/backup/run/Executes immediate Alation database backup.
89Restore Database BackupPOST /api/v1/admin/restore/Restores Alation metadata store from backup file.
90Get License QuotasGET /api/v1/admin/license/Inspects licensed user seats and connector quotas.
91Update License KeyPOST /api/v1/admin/license/update/Applies new enterprise license key file.
92Get Network SettingsGET /api/v1/admin/network/Inspects proxy and SSL certificate settings.
93Update SSL CertificatePOST /api/v1/admin/ssl/update/Uploads custom TLS/SSL certificate.
94Get SMTP SettingsGET /api/v1/admin/email/config/Inspects outbound email notification server.
95Update SMTP SettingsPUT /api/v1/admin/email/config/Modifies SMTP host, port, and credentials.
96Send Test EmailPOST /api/v1/admin/email/test/Sends test email to verify SMTP gateway.
97Get SSO ConfigurationGET /api/v1/admin/sso/config/Inspects SAML 2.0 / OIDC identity settings.
98Update SSO ConfigurationPUT /api/v1/admin/sso/config/Updates IdP metadata URL or SSO entity ID.
99Flush Query CachePOST /api/v1/admin/cache/flush/Clears internal query and metadata cache.
100Get Container HealthGET /api/v1/admin/health/Verifies Docker/k8s container service health.
101Purge Catalog AssetDELETE /integration/v1/trash/{id}/purge/Permanently purges archived catalog object.
102Get Data Source Connection StatusGET /integration/v1/datasource/{id}/status/Checks live connection status to database host.
103Get Data Source Extraction ScheduleGET /integration/v1/datasource/{id}/schedule/Inspects automated metadata extraction schedule.
104Set Data Source Extraction SchedulePUT /integration/v1/datasource/{id}/schedule/Configures cron schedule for metadata extraction.
105Get OCF Connector VersionGET /integration/v1/ocf/connector/{id}/Inspects installed Open Connector Framework driver version.
106Upgrade OCF ConnectorPOST /integration/v1/ocf/connector/{id}/upgrade/Upgrades OCF connector plugin to latest version.
107Get Table Row CountGET /integration/v1/table/{id}/row_count/Fetches cached or live table row count.
108Get Table Popularity ScoreGET /integration/v1/table/{id}/popularity/Inspects usage popularity score based on query logs.
109Get Column Top ValuesGET /integration/v1/column/{id}/top_values/Fetches top frequent distinct values for column.
110Get Column Value DistributionGET /integration/v1/column/{id}/histogram/Fetches value distribution histogram for numeric column.
111Get Article RevisionsGET /integration/v1/article/{id}/revisions/Lists historical revision edits for wiki page.
112Revert Article RevisionPOST /integration/v1/article/{id}/revert/{rev_id}/Rolls back article content to previous revision.
113Get Article ChildrenGET /integration/v1/article/{id}/children/Lists child articles nested under parent article.
114Get Query Execution HistoryGET /integration/v1/query/{id}/history/Inspects past execution runs of saved query.
115Get Query EndorsementsGET /integration/v1/query/{id}/endorsements/Lists stewards who endorsed or deprecated query.
116Get BI Folder ListGET /integration/v1/bi_folders/Lists folders in connected Tableau/PowerBI catalog.
117Get BI Workbooks ListGET /integration/v1/bi_workbooks/Lists workbooks in connected BI server.
118Get BI Datasources ListGET /integration/v1/bi_datasources/Lists datasources used by BI dashboards.
119Get BI Fields ListGET /integration/v1/bi_fields/Lists calculated fields used in BI reports.
120Get Glossary HierarchyGET /integration/v1/glossary/tree/Fetches full business term dictionary tree.
121Get Term Related TermsGET /integration/v1/glossary/terms/{id}/related/Lists terms linked as related concepts.
122Link Related TermsPOST /integration/v1/glossary/terms/{id}/link_related/Links two business glossary terms together.
123Get Term SynonymsGET /integration/v1/glossary/terms/{id}/synonyms/Lists alternate acronyms and synonyms for term.
124Add Term SynonymPOST /integration/v1/glossary/terms/{id}/add_synonym/Adds synonym to business glossary term.
125Get Stewardship Scores by SchemaGET /integration/v1/stewardship/schema/{id}/Evaluates documentation completeness for schema.
126Get Stewardship Top ContributorsGET /integration/v1/stewardship/leaderboard/Lists top active data stewards by edits.
127Get Job Log TraceGET /api/v1/job/{job_id}/log/Downloads execution log trace for background task.
128Get Export Task StatusGET /integration/v1/export/task/{id}/Inspects status of background CSV export task.
129Get Import Task StatusGET /integration/v1/import/task/{id}/Inspects status of background CSV import task.
130Get Group MembersGET /integration/v1/group/{id}/members/Lists all user accounts in group.
131Get Group RolesGET /integration/v1/group/{id}/roles/Lists administrative roles assigned to group.
132Assign Role to GroupPOST /integration/v1/group/{id}/assign_role/Grants administrative role to group.
133Revoke Role from GroupPOST /integration/v1/group/{id}/revoke_role/Revokes administrative role from group.
134Get Access Request ListGET /integration/v1/access_request/Lists pending data access requests submitted by users.
135Approve Access RequestPOST /integration/v1/access_request/{id}/approve/Approves data access request ticket.
136Deny Access RequestPOST /integration/v1/access_request/{id}/deny/Rejects data access request ticket.
137Get Access Request DetailsGET /integration/v1/access_request/{id}/Fetches justification and details for access request.
138Get Policy Rule ListGET /integration/v1/policy_rule/Lists active data governance rules.
139Create Policy RulePOST /integration/v1/policy_rule/Provisions data masking or restricted access policy rule.
140Delete Policy RuleDELETE /integration/v1/policy_rule/{id}/Removes data governance policy rule.
141Get User Bookmark ListGET /integration/v1/user/{id}/bookmarks/Lists catalog assets bookmarked by user.
142Add BookmarkPOST /integration/v1/user/bookmark/Bookmarks catalog asset for user.
143Remove BookmarkDELETE /integration/v1/user/bookmark/{id}/Removes bookmarked catalog asset.
144Get User Search HistoryGET /integration/v1/user/{id}/search_history/Inspects recent search queries typed by user.
145Clear User Search HistoryDELETE /integration/v1/user/{id}/search_history/Purges search query history for user.
146Get Compose Execution SettingsGET /api/v1/compose/settings/Inspects row limits and timeouts for SQL execution.
147Set Compose Execution SettingsPUT /api/v1/compose/settings/Updates row limits and timeouts for SQL execution.
148Get Compose Connection ListGET /api/v1/compose/connections/Lists database connections available in Compose.
149Get Lexicon Extracted TermsGET /integration/v1/lexicon/terms/Lists terms automatically mined from query logs.
150Approve Lexicon TermPOST /integration/v1/lexicon/terms/{id}/approve/Promotes auto-mined term to official glossary.
151Reject Lexicon TermPOST /integration/v1/lexicon/terms/{id}/reject/Discards auto-mined term.
152Get System Config ParameterGET /api/v1/admin/config/{key}/Inspects Alation system configuration parameter.
153Set System Config ParameterPUT /api/v1/admin/config/{key}/Updates Alation system configuration parameter.
154Get Database Migration StatusGET /api/v1/admin/db_migration/status/Inspects Django database migration state.
155Run Database MigrationPOST /api/v1/admin/db_migration/run/Executes pending database migrations.
156Get Celery Worker StatusGET /api/v1/admin/celery/status/Inspects active Celery background task workers.
157Restart Celery WorkerPOST /api/v1/admin/celery/restart/Restarts Celery background task worker process.
158Get Redis Cache StatusGET /api/v1/admin/redis/status/Inspects internal Redis cache utilization.
159Flush Redis CachePOST /api/v1/admin/redis/flush/Flushes internal Redis cache memory.
160Get Search Index StatusGET /api/v1/admin/search/status/Inspects status of internal search index.
161Optimize Search IndexPOST /api/v1/admin/search/optimize/Compacts internal search index storage.
162Get Memory Usage MetricsGET /api/v1/admin/metrics/memory/Pulls RAM usage breakdown across containers.
163Get CPU Usage MetricsGET /api/v1/admin/metrics/cpu/Pulls CPU utilization metrics.
164Get Disk Storage MetricsGET /api/v1/admin/metrics/disk/Pulls disk space usage per partition.
165Get Audit Trail LogGET /api/v1/admin/audit_log/Exports administrative event audit log.
166Get LDAP Config StatusGET /api/v1/admin/ldap/status/Inspects LDAP directory integration status.
167Test LDAP ConnectionPOST /api/v1/admin/ldap/test/Tests LDAP server connection and authentication.
168Get SAML Config StatusGET /api/v1/admin/saml/status/Inspects SAML 2.0 SSO integration status.
169Test SAML ConfigurationPOST /api/v1/admin/saml/test/Tests SAML IdP metadata and assertion parsing.
170Get Mail Server ConfigGET /api/v1/admin/email/status/Inspects outbound SMTP server health.
171Get SSL Certificate DetailsGET /api/v1/admin/ssl/info/Inspects TLS/SSL certificate expiration date.
172Get License Seat UsageGET /api/v1/admin/license/usage/Inspects assigned active user seats vs quota.
173Check Alation System VersionGET /api/v1/admin/version/Outputs Alation release version and build hash.
174Check Alation API PingGET /api/v1/ping/Health check endpoint returning 200 OK.
175Get Column Data TypesGET /integration/v1/column/{id}/data_type/Inspects column native and logical data types.
176Get Table Foreign KeysGET /integration/v1/table/{id}/foreign_keys/Lists foreign key relationships for table.
177Get Table Primary KeyGET /integration/v1/table/{id}/primary_key/Identifies primary key columns for table.
178Get Table IndexesGET /integration/v1/table/{id}/indexes/Lists secondary indexes on table.
179Get Schema Table CountGET /integration/v1/schema/{id}/table_count/Counts total tables in schema.
180Get Schema View CountGET /integration/v1/schema/{id}/view_count/Counts total views in schema.
181Get Data Source Table CountGET /integration/v1/datasource/{id}/table_count/Counts total tables across data source.
182Get Data Source Query CountGET /integration/v1/datasource/{id}/query_count/Counts total saved queries for data source.
183Get Data Source User CountGET /integration/v1/datasource/{id}/user_count/Counts distinct users querying data source.
184Get User Recent QueriesGET /integration/v1/user/{id}/recent_queries/Lists recent queries executed by user.
185Get User Favorite QueriesGET /integration/v1/user/{id}/favorite_queries/Lists favorite queries bookmarked by user.
186Get User Published ArticlesGET /integration/v1/user/{id}/articles/Lists wiki documentation pages published by user.
187Get User Stewardship AssetsGET /integration/v1/user/{id}/stewardship_assets/Lists catalog assets mentored by user.
188Get Article AttachmentsGET /integration/v1/article/{id}/attachments/Lists files attached to wiki page.
189Upload Article AttachmentPOST /integration/v1/article/{id}/upload_attachment/Attaches file payload to wiki page.
190Delete Article AttachmentDELETE /integration/v1/article/{id}/attachment/{att_id}/Removes attached file from wiki page.
191Get Glossary Term ChildrenGET /integration/v1/glossary/terms/{id}/children/Lists child terms nested under business term.
192Get Glossary Term ParentsGET /integration/v1/glossary/terms/{id}/parents/Lists parent terms governing business term.
193Get Glossary Term AssetsGET /integration/v1/glossary/terms/{id}/assets/Lists physical columns and tables mapped to term.
194Link Term to TablePOST /integration/v1/glossary/mapping/table/Associates business term with physical table.
195Unlink Term from TableDELETE /integration/v1/glossary/mapping/table/{id}/Removes association between term and table.
196Unlink Term from ColumnDELETE /integration/v1/glossary/mapping/column/{id}/Removes association between term and column.
197Get Group User CountGET /integration/v1/group/{id}/user_count/Counts total user accounts in group.
198Check API Authentication StatusGET /integration/v1/auth/status/Verifies API session token validity.
199Check API Version StringGET /integration/v1/version/Outputs Alation API specification version string.
200Check System Health REST APIGET /api/v1/health/Queries HTTP REST endpoint for server health.

PostgreSQL OLTP

Technical Architecture & Overview

PostgreSQL is an advanced, enterprise-grade open-source object-relational database management system (ORDBMS). It features full ACID compliance, multi-version concurrency control (MVCC), extensible data types (JSONB, PostGIS spatial, Vector, HSTORE), and sophisticated indexing mechanisms (B-Tree, GIN, GiST, BRIN, SP-GiST).

Primary Use Cases: Mission-critical OLTP applications, geo-spatial systems (PostGIS), AI vector search (pgvector), complex transactional financial platforms, and hybrid relational-document stores.

Core Components: Postmaster process, Shared Buffers, Write-Ahead Log (WAL), Background Writer, Auto-vacuum daemon, and Replication Slots (Streaming & Logical CDC).

Exhaustive 200-command administrative, diagnostic, and operational matrix for PostgreSQL.

#Operation / ActionSQL / CLI SyntaxDescription
1Start Servicepg_ctl -D /var/lib/postgresql/data startBoots PostgreSQL postmaster process.
2Stop Servicepg_ctl -D /var/lib/postgresql/data stop -m fastSafely shuts down database instance.
3Interactive CLIpsql -U postgres -d enterprise_dbConnects to transactional database.
4Backup Databasepg_dump -Fc -v enterprise_db -f backup.dumpGenerates compressed custom archive.
5Restore Databasepg_restore -d enterprise_db backup.dumpRestores archive into target DB.
6Vacuum MaintenanceVACUUM ANALYZE VERBOSE;Reclaims space and updates stats.
7Reindex TablesREINDEX DATABASE CONCURRENTLY enterprise_db;Rebuilds bloated database indexes.
8List Active QueriesSELECT pid, query, state FROM pg_stat_activity;Inspects running database operations.
9Terminate QuerySELECT pg_cancel_backend(pid);Stops a long-running transaction.
10Create RoleCREATE ROLE app_user WITH LOGIN PASSWORD 'secure';Provisions a new database login.
11Grant PrivilegesGRANT ALL PRIVILEGES ON DATABASE enterprise_db TO app_user;Assigns complete access rights.
12Create DatabaseCREATE DATABASE enterprise_db ENCODING 'UTF8';Provisions new transactional DB.
13Drop DatabaseDROP DATABASE IF EXISTS enterprise_db;Deletes database instance.
14Explain Query PlanEXPLAIN ANALYZE SELECT * FROM users;Evaluates execution plan performance.
15Check Table SizesSELECT pg_size_pretty(pg_total_relation_size('users'));Calculates total table disk usage.
16Show ConfigurationSHOW max_connections;Inspects server runtime parameter.
17Reload ConfigSELECT pg_reload_conf();Reloads postgresql.conf changes.
18Create ExtensionCREATE EXTENSION IF NOT EXISTS "uuid-ossp";Provisions cryptographic extension.
19List ExtensionsSELECT * FROM pg_available_extensions;Inspects installed modules.
20Check Replication LagSELECT client_addr, write_lag FROM pg_stat_replication;Monitors standby sync delay.
21Create Table PartitionCREATE TABLE logs_y2026m07 PARTITION OF logs FOR VALUES FROM ('2026-07-01') TO ('2026-08-01');Provisions time-series partition.
22Alter Table SchemaALTER TABLE users ADD COLUMN email_verified BOOLEAN DEFAULT FALSE;Modifies physical table schema.
23Create IndexCREATE INDEX CONCURRENTLY idx_users_email ON users(email);Builds index without locking table.
24Analyze Dead TuplesSELECT schemaname, relname, n_dead_tup FROM pg_stat_user_tables;Checks table bloat status.
25Check Lock ContentionSELECT * FROM pg_locks l JOIN pg_stat_activity a ON l.pid = a.pid;Inspects blocking locks.
26Set Statement TimeoutSET statement_timeout = '30s';Enforces query execution limit.
27Create SchemaCREATE SCHEMA staging;Provisions logical namespace.
28Show Disk UsageSELECT pg_size_pretty(pg_database_size('enterprise_db'));Measures total database storage.
29Verify CheckpointsCHECKPOINT;Forces immediate dirty buffer flush.
30Cluster TableCLUSTER users USING users_pkey;Physically reorders table by index.
31Kill Backend ProcessSELECT pg_terminate_backend(pid);Hard-terminates connection and transaction immediately.
32Rotate Log FilesSELECT pg_rotate_logfile();Forces PostgreSQL logger to switch to new log file.
33Promote Standby Serverpg_ctl promote -D /var/lib/postgresql/dataPromotes read-only standby node to primary master.
34Pause WAL ReplaySELECT pg_wal_replay_pause();Pauses WAL recovery on standby replica server.
35Resume WAL ReplaySELECT pg_wal_replay_resume();Resumes WAL recovery replay on standby replica.
36Check WAL Replay StatusSELECT pg_is_wal_replay_paused();Checks if WAL replay is currently paused on standby.
37Get Physical WAL PositionSELECT pg_current_wal_lsn();Returns current write-ahead log byte LSN location.
38Create Replication SlotSELECT * FROM pg_create_physical_replication_slot('standby_1');Provisions streaming replication slot.
39Drop Replication SlotSELECT pg_drop_replication_slot('standby_1');Deletes physical streaming replication slot.
40List Replication SlotsSELECT * FROM pg_replication_slots;Inspects active logical and physical replication slots.
41Create Logical Replication SlotSELECT * FROM pg_create_logical_replication_slot('sub_1', 'pgoutput');Provisions logical decoding slot for CDC.
42Create PublicationCREATE PUBLICATION app_pub FOR TABLE users, orders;Provisions publication for logical CDC streaming.
43Alter PublicationALTER PUBLICATION app_pub ADD TABLE products;Adds table to logical replication publication.
44Drop PublicationDROP PUBLICATION app_pub;Removes logical replication publication.
45Create SubscriptionCREATE SUBSCRIPTION app_sub CONNECTION 'host=10.0.0.1 dbname=enterprise_db' PUBLICATION app_pub;Provisions logical subscription node.
46Alter SubscriptionALTER SUBSCRIPTION app_sub REFRESH PUBLICATION;Refreshes publication schema on subscriber node.
47Drop SubscriptionDROP SUBSCRIPTION app_sub;Removes logical replication subscription.
48Vacuum Full TableVACUUM FULL VERBOSE users;Locks table and rebuilds physical relation storage.
49Analyze Specific TableANALYZE VERBOSE users;Gathers column statistics for specific table.
50List Database UsersSELECT usename, valuntil FROM pg_user;Inspects database accounts and password expiry dates.
51Change User PasswordALTER USER app_user WITH PASSWORD 'NewSecurePass123!';Updates authentication password for database user.
52Lock User AccountALTER ROLE app_user NOLOGIN;Disables login privileges for a role.
53Unlock User AccountALTER ROLE app_user LOGIN;Enables login privileges for a role.
54Grant Schema UsageGRANT USAGE ON SCHEMA staging TO app_user;Grants namespace usage access to role.
55Grant Table SelectGRANT SELECT, INSERT ON ALL TABLES IN SCHEMA public TO app_user;Grants table access rights in schema.
56Revoke Table PrivilegesREVOKE ALL PRIVILEGES ON TABLE users FROM app_user;Removes table permissions from role.
57Set Default PrivilegesALTER DEFAULT PRIVILEGES IN SCHEMA public GRANT SELECT ON TABLES TO app_user;Enforces automatic grants on future tables.
58Show Connection LimitSHOW max_connections;Displays max allowed concurrent client connections.
59Set Work MemorySET work_mem = '64MB';Increases per-operation sort and hash memory for session.
60Set Maintenance Work MemorySET maintenance_work_mem = '512MB';Increases RAM for VACUUM, CREATE INDEX, and ALTER TABLE.
61Show Shared BuffersSHOW shared_buffers;Displays RAM size allocated for shared buffer cache.
62Check Cache Hit RatioSELECT sum(heap_blks_hit) / (sum(heap_blks_hit) + sum(heap_blks_read)) FROM pg_statio_user_tables;Calculates database buffer cache hit ratio.
63Check Index Hit RatioSELECT sum(idx_blks_hit) / (sum(idx_blks_hit) + sum(idx_blks_read)) FROM pg_statio_user_indexes;Calculates index buffer cache hit ratio.
64List Unused IndexesSELECT relname, indexrelname, idx_scan FROM pg_stat_user_indexes WHERE idx_scan = 0;Identifies candidates for index removal.
65List Duplicate IndexesSELECT pg_size_pretty(sum(pg_relation_size(idx))) FROM pg_stat_user_indexes;Inspects memory overhead of secondary indexes.
66Drop IndexDROP INDEX CONCURRENTLY idx_users_email;Removes index without locking concurrent writes.
67Create Partial IndexCREATE INDEX idx_active_users ON users(id) WHERE status = 'active';Builds filtered index for active rows only.
68Create Expression IndexCREATE INDEX idx_lower_email ON users(LOWER(email));Builds index on functional string transformation.
69Create GIN IndexCREATE INDEX idx_data_gin ON log_entries USING GIN(payload);Provisions Generalized Inverted Index for JSONB.
70Create GiST IndexCREATE INDEX idx_geom_gist ON locations USING GIST(geom);Provisions Generalized Search Tree for PostGIS geometries.
71Create BRIN IndexCREATE INDEX idx_created_brin ON timeseries USING BRIN(created_at);Provisions Block Range Index for append-only data.
72Check Table BloatSELECT schemaname, tblname, bs*tblpages AS real_size FROM pg_tables;Estimates physical relation storage bloat.
73List Prepared TransactionsSELECT * FROM pg_prepared_xacts;Inspects unresolved two-phase commit transactions.
74Rollback Prepared TransactionROLLBACK PREPARED 'gid_12345';Aborts an orphaned 2PC transaction.
75Commit Prepared TransactionCOMMIT PREPARED 'gid_12345';Finalizes a 2PC transaction.
76Create TablespaceCREATE TABLESPACE fast_ssd LOCATION '/mnt/disks/ssd1';Provisions physical storage tablespace.
77Move Table to TablespaceALTER TABLE users SET TABLESPACE fast_ssd;Migrates physical table storage to new disk.
78Move Index to TablespaceALTER INDEX idx_users_email SET TABLESPACE fast_ssd;Migrates index storage to new disk.
79Drop TablespaceDROP TABLESPACE fast_ssd;Deletes empty storage tablespace.
80Check Table LocksSELECT relation::regclass, mode, granted FROM pg_locks WHERE NOT granted;Lists queries waiting for table locks.
81Set Lock TimeoutSET lock_timeout = '5s';Aborts statement if lock cannot be acquired in 5s.
82Set Idle In Transaction TimeoutSET idle_in_transaction_session_timeout = '10s';Terminates sessions idling inside open transaction.
83Create Foreign Data WrapperCREATE EXTENSION postgres_fdw;Installs extension for querying external DBs.
84Create Foreign ServerCREATE SERVER remote_pg FOREIGN DATA WRAPPER postgres_fdw OPTIONS (host '10.0.0.2', dbname 'remote_db');Defines remote server endpoint.
85Create User MappingCREATE USER MAPPING FOR local_user SERVER remote_pg OPTIONS (user 'remote_user', password 'pass');Maps local role to remote server login.
86Import Foreign SchemaIMPORT FOREIGN SCHEMA public FROM SERVER remote_pg INTO remote_staging;Bulk imports remote table definitions.
87Create Materialized ViewCREATE MATERIALIZED VIEW mv_user_stats AS SELECT dept, COUNT(*) FROM users GROUP BY dept;Provisions disk-backed view snapshot.
88Refresh Materialized ViewREFRESH MATERIALIZED VIEW CONCURRENTLY mv_user_stats;Refreshes view data without blocking queries.
89Drop Materialized ViewDROP MATERIALIZED VIEW mv_user_stats;Removes materialized view.
90Create Event TriggerCREATE EVENT TRIGGER ddl_logger ON ddl_command_start EXECUTE FUNCTION log_ddl();Fires function on DDL schema changes.
91Disable Event TriggerALTER EVENT TRIGGER ddl_logger DISABLE;Deactivates DDL event trigger.
92Drop Event TriggerDROP EVENT TRIGGER ddl_logger;Deletes DDL event trigger.
93List Database TriggersSELECT tgname, relname FROM pg_trigger t JOIN pg_class r ON t.tgrelid = r.oid;Lists active table triggers.
94Disable Table TriggersALTER TABLE users DISABLE TRIGGER ALL;Disables trigger execution during bulk load.
95Enable Table TriggersALTER TABLE users ENABLE TRIGGER ALL;Re-enables table trigger execution.
96Create Advisory LockSELECT pg_advisory_lock(123456);Acquires application-level lock ID.
97Release Advisory LockSELECT pg_advisory_unlock(123456);Releases application-level lock ID.
98Unlock All Advisory LocksSELECT pg_advisory_unlock_all();Releases all held session advisory locks.
99Dump Schema Onlypg_dump -s -d enterprise_db -f schema.sqlExports database structure without table data.
100Dump Data Onlypg_dump -a -d enterprise_db -f data.sqlExports table data rows without DDL schema.
101Check Table FillfactorSELECT relname, reloptions FROM pg_class WHERE reloptions IS NOT NULL;Inspects custom storage fillfactor settings.
102Set Table FillfactorALTER TABLE users SET (fillfactor = 80);Leaves 20% free space per page for HOT updates.
103Check HOT Updates RatioSELECT n_tup_upd, n_tup_hot_upd FROM pg_stat_user_tables;Measures Heap-Only Tuple update efficiency.
104Check WAL Disk UsageSELECT pg_size_pretty(sum(size)) FROM pg_ls_waldir();Calculates disk space consumed by WAL files.
105Force Log RotationSELECT pg_rotate_logfile();Forces database log file rotation.
106Check Foreign Key IndexesSELECT conname, conrelid::regclass FROM pg_constraint WHERE contype = 'f';Lists foreign keys to verify indexing.
107Check Table Toast SizeSELECT pg_size_pretty(pg_relation_size(reltoastrelid)) FROM pg_class WHERE relname = 'users';Calculates TOAST out-of-line storage size.
108Check Database EncodingSELECT datname, pg_encoding_to_char(encoding) FROM pg_database;Inspects database character encoding.
109Check Active Extensions VersionSELECT extname, extversion FROM pg_extension;Lists installed extensions and versions.
110Set Session TimezoneSET TIME ZONE 'UTC';Sets active session time zone to UTC.
111Show Active TimezoneSHOW TIMEZONE;Outputs current session time zone.
112Check Database CollationSELECT datname, datcollate FROM pg_database;Inspects character collation order.
113Check Background Writer StatsSELECT * FROM pg_stat_bgwriter;Monitors checkpoint and bgwriter buffer flushes.
114Check Buffer Pool Usage per TableSELECT c.relname, count(*) AS buffers FROM pg_buffercache b JOIN pg_class c ON b.relfilenode = c.relfilenode GROUP BY c.relname;Inspects shared buffers usage per table.
115Check Sequence Current ValueSELECT last_value FROM user_id_seq;Inspects current sequence counter value.
116Set Sequence ValueSELECT setval('user_id_seq', 5000);Updates sequence counter value.
117Check Table Storage EncodingsSELECT attname, attstorage FROM pg_attribute WHERE attrelid = 'users'::regclass;Inspects TOAST compression modes (PLAIN, EXTENDED, MAIN).
118Set Column Storage CompressionALTER TABLE users ALTER COLUMN payload SET STORAGE EXTENDED;Enforces main and TOAST storage compression.
119Check Auto-vacuum ActivitySELECT * FROM pg_stat_progress_vacuum;Monitors active running vacuum jobs.
120Check CREATE INDEX ProgressSELECT * FROM pg_stat_progress_create_index;Monitors progress of concurrent index builds.
121Check ANALYZE ProgressSELECT * FROM pg_stat_progress_analyze;Monitors active table analyze jobs.
122Check CLUSTER ProgressSELECT * FROM pg_stat_progress_cluster;Monitors active cluster/vacuum full jobs.
123Set Autovacuum Scale FactorALTER TABLE users SET (autovacuum_vacuum_scale_factor = 0.05);Triggers vacuum when 5% rows change.
124Set Autovacuum ThresholdALTER TABLE users SET (autovacuum_vacuum_threshold = 100);Sets base dead tuple threshold for vacuum.
125Check Replication Slot LagSELECT slot_name, pg_wal_lsn_diff(pg_current_wal_lsn(), restart_lsn) FROM pg_replication_slots;Calculates LSN lag bytes for replication slots.
126Get Transaction ID AgeSELECT datname, age(datfrozenxid) FROM pg_database;Monitors transaction ID wraparound age.
127Check Multixact AgeSELECT datname, mxid_age(datminmxid) FROM pg_database;Monitors multixact ID wraparound age.
128Set Statement Timeout GlobalALTER DATABASE enterprise_db SET statement_timeout = '60s';Caps statement execution time globally.
129Set Idle In Transaction Timeout GlobalALTER DATABASE enterprise_db SET idle_in_transaction_session_timeout = '30s';Caps idle transaction time globally.
130Check Server VersionSELECT version();Outputs running PostgreSQL release and build version.
131Check Index Usage DetailsSELECT schemaname, relname, indexrelname, idx_scan, idx_tup_read, idx_tup_fetch FROM pg_stat_user_indexes;Inspects index scan efficiency.
132Check Table I/O StatisticsSELECT schemaname, relname, heap_blks_read, heap_blks_hit FROM pg_statio_user_tables;Measures table buffer cache hit ratio.
133Check Index I/O StatisticsSELECT schemaname, relname, indexrelname, idx_blks_read, idx_blks_hit FROM pg_statio_user_indexes;Measures index buffer cache hit ratio.
134Check Database Size BytesSELECT pg_database_size('enterprise_db');Outputs exact database size in bytes.
135Check Table Size BytesSELECT pg_relation_size('users');Outputs table data size in bytes without indexes.
136Check Total Relation Size BytesSELECT pg_total_relation_size('users');Outputs table data plus index size in bytes.
137Check Index Size BytesSELECT pg_relation_size('idx_users_email');Outputs index size in bytes.
138Check TOAST Size BytesSELECT pg_total_relation_size(reltoastrelid) FROM pg_class WHERE relname='users';Outputs TOAST table size in bytes.
139Check System Catalogs ListSELECT relname FROM pg_class WHERE relnamespace='pg_catalog'::regnamespace AND relkind='r';Lists PostgreSQL system catalog tables.
140Check System Views ListSELECT viewname FROM pg_views WHERE schemaname='pg_catalog';Lists PostgreSQL system catalog views.
141Check Active Replication OriginsSELECT * FROM pg_replication_origin_status;Inspects active logical replication origin LSNs.
142Create Replication OriginSELECT pg_replication_origin_create('origin1');Provisions logical replication origin slot.
143Drop Replication OriginSELECT pg_replication_origin_drop('origin1');Deletes logical replication origin slot.
144Check Extension CommandsSELECT * FROM pg_available_extension_versions WHERE name='uuid-ossp';Inspects available versions of extension.
145Alter Extension UpdateALTER EXTENSION "uuid-ossp" UPDATE TO '1.1';Upgrades extension version in active database.
146Drop ExtensionDROP EXTENSION IF EXISTS "uuid-ossp";Removes extension from database.
147Check Setting SourceSELECT name, setting, source FROM pg_settings WHERE source <> 'default';Lists non-default server parameters.
148Set Parameter Session LocalSET LOCAL work_mem = '128MB';Sets parameter for duration of active transaction.
149Reset Parameter SessionRESET work_mem;Restores parameter to default setting.
150Reset All Session ParametersRESET ALL;Restores all session parameters to default settings.
151Check User Roles MembershipSELECT r.rolname, m.roleid::regrole FROM pg_roles r JOIN pg_auth_members m ON r.oid=m.member;Lists role memberships.
152Grant Role to RoleGRANT data_analyst TO app_user;Assigns role permissions to user account.
153Revoke Role from RoleREVOKE data_analyst FROM app_user;Revokes role permissions from user account.
154Check Default Privileges ListSELECT * FROM pg_default_acl;Lists schema-level default access control settings.
155Check Database Triggers ListSELECT tgname, relname FROM pg_trigger JOIN pg_class ON pg_trigger.tgrelid=pg_class.oid;Lists active database triggers.
156Check Database Locks SummarySELECT mode, count(*) FROM pg_locks GROUP BY mode;Groups active locks by lock mode.
157Check Database Process MemorySELECT pid, usename, query, state FROM pg_stat_activity WHERE state <> 'idle';Inspects active processing backends.
158Check Database Transaction AgeSELECT datname, age(datfrozenxid) FROM pg_database WHERE datname='enterprise_db';Monitors database transaction ID age.
159Check Multixact ID AgeSELECT datname, mxid_age(datminmxid) FROM pg_database WHERE datname='enterprise_db';Monitors database multixact ID age.
160Check Log File DirectorySELECT pg_ls_dir('log');Lists log file names in log directory.
161Read Log File PortionSELECT pg_read_file('postgresql.log', 0, 1000);Reads raw text segment from log file.
162Check Stat Clear SnapshotSELECT pg_stat_clear_snapshot();Flushes cached statistics snapshot.
163Check Stat ResetSELECT pg_stat_reset();Resets server-wide statistics counters.
164Check Stat Reset SharedSELECT pg_stat_reset_shared('bgwriter');Resets shared subsystem statistics counters.
165Check Database System IdentifierSELECT system_identifier FROM pg_control_system();Outputs unique system cluster ID.
166Check Database Control File StateSELECT state, checkpoint_lsn FROM pg_control_checkpoint();Inspects control file state and last LSN.
167Check Database Recovery StatusSELECT pg_is_in_recovery();Returns true if database is running in standby/recovery mode.
168Check Database Last WAL Receive LSNSELECT pg_last_wal_receive_lsn();Outputs LSN location of last WAL received by standby.
169Check Database Last WAL Replay LSNSELECT pg_last_wal_replay_lsn();Outputs LSN location of last WAL replayed by standby.
170Check Database Last WAL Replay TimeSELECT pg_last_xact_replay_timestamp();Outputs timestamp of last transaction replayed by standby.
171Check Database WAL Keep SegmentsSHOW wal_keep_size;Inspects WAL retain size setting for replication.
172Set Database WAL Keep SegmentsALTER SYSTEM SET wal_keep_size = '10GB';Sets 10GB WAL retain threshold for standby sync.
173Check Database Max WAL SizeSHOW max_wal_size;Inspects max WAL size setting before checkpoint.
174Set Database Max WAL SizeALTER SYSTEM SET max_wal_size = '16GB';Configures 16GB max WAL size before checkpoint.
175Check Database Min WAL SizeSHOW min_wal_size;Inspects minimum retained WAL size.
176Check Database Checkpoint TimeoutSHOW checkpoint_timeout;Inspects automatic checkpoint time window.
177Set Database Checkpoint TimeoutALTER SYSTEM SET checkpoint_timeout = '15min';Sets 15-minute automatic checkpoint interval.
178Check Database Checkpoint Completion TargetSHOW checkpoint_completion_target;Inspects checkpoint write spread factor.
179Set Database Checkpoint Completion TargetALTER SYSTEM SET checkpoint_completion_target = 0.9;Spreads checkpoint I/O over 90% of window.
180Check Database Archive ModeSHOW archive_mode;Inspects continuous WAL archive mode.
181Check Database Archive CommandSHOW archive_command;Inspects script command used for WAL archiving.
182Check Database Archive TimeoutSHOW archive_timeout;Inspects maximum duration before forcing WAL archive segment.
183Set Database Archive TimeoutALTER SYSTEM SET archive_timeout = '300s';Forces WAL segment archive every 5 minutes.
184Check Database Archive StatusSELECT * FROM pg_stat_archiver;Monitors total archived and failed WAL segments.
185Check Database SSL StatusSHOW ssl;Returns on/off for SSL connection encryption.
186Check Database SSL CiphersSHOW ssl_ciphers;Lists active SSL cipher algorithms.
187Check Database Password EncryptionSHOW password_encryption;Inspects password hashing algorithm (scram-sha-256).
188Check Database Client EncodingSHOW client_encoding;Displays active client character encoding.
189Set Database Client EncodingSET client_encoding = 'UTF8';Configures client character encoding.
190Check Database Server Version StringSELECT version();Outputs complete version, compiler, and build architecture string.
191Check Database Ping ConnectionSELECT 1;Simple database ping test returning 1.
192Check Database Current DateSELECT CURRENT_DATE;Returns current server date.
193Check Database Current TimeSELECT CURRENT_TIME;Returns current server time with timezone.
194Check Database Current TimestampSELECT CURRENT_TIMESTAMP;Returns current server timestamp.
195Check Database Session UserSELECT SESSION_USER;Returns session user account.
196Check Database Current UserSELECT CURRENT_USER;Returns current execution user account.
197Check Database Current SchemaSELECT CURRENT_SCHEMA();Returns current active schema name.
198Check Database Current Search PathSHOW search_path;Displays active schema search path.
199Set Database Search PathSET search_path TO staging, public;Configures active schema search path.
200Check Database UptimeSELECT pg_postmaster_start_time();Outputs exact postmaster startup timestamp.

MySQL Enterprise OLTP

Technical Architecture & Overview

MySQL Enterprise is the industry's premier open-source relational database management system, powered by the ACID-compliant InnoDB storage engine. It features thread-based concurrency, pluggable storage engines (InnoDB, NDB Cluster, Memory), and robust high-availability architectures (Group Replication, InnoDB Cluster, Source-Replica GTID replication).

Primary Use Cases: High-volume web applications, e-commerce transaction processing, SaaS backends, scale-out read-heavy microservices, and managed cloud OLTP databases.

Core Components: SQL Parser/Optimizer, InnoDB Buffer Pool, Redo Log, Undo Log, Doublewrite Buffer, Adaptive Hash Index, and Binary Logs (GTID CDC).

Exhaustive 200-command administrative, diagnostic, and operational matrix for MySQL Enterprise.

#Operation / ActionSQL / CLI SyntaxDescription
1Start Daemonsudo systemctl start mysqldStarts MySQL server daemon.
2Stop Daemonsudo systemctl stop mysqldStops MySQL server daemon.
3Interactive Shellmysql -u root -pOpens administrative SQL monitor.
4Full Database Backupmysqldump -u root -p --all-databases > backup.sqlExports entire cluster data dump.
5Restore Databasemysql -u root -p enterprise_db < backup.sqlRestores SQL dump file.
6InnoDB Status CheckSHOW ENGINE INNODB STATUS\GInspects storage engine metrics.
7Process List InspectionSHOW FULL PROCESSLIST;Views active query threads.
8Kill Process ThreadKILL 4512;Terminates hung query thread.
9Create User AccountCREATE USER 'app'@'localhost' IDENTIFIED BY 'Pass!';Provisions new MySQL user.
10Grant PrivilegesGRANT ALL PRIVILEGES ON enterprise_db.* TO 'app'@'localhost';Assigns database access.
11Flush TablesFLUSH TABLES WITH READ LOCK;Locks all tables for backup.
12Release Read LocksUNLOCK TABLES;Releases table locks.
13Analyze TableANALYZE TABLE users;Updates index cardinality stats.
14Optimize TableOPTIMIZE TABLE users;Defragments InnoDB table space.
15Check Table IntegrityCHECK TABLE users;Scans table for corruption.
16Show Global StatusSHOW GLOBAL STATUS LIKE 'Conn%';Inspects connection metrics.
17Show Global VariablesSHOW VARIABLES LIKE 'innodb_buffer_pool_size';Inspects memory configurations.
18Explain Query ExecutionEXPLAIN FORMAT=JSON SELECT * FROM users;Performs deep query analysis.
19Create DatabaseCREATE DATABASE enterprise_db CHARACTER SET utf8mb4;Provisions new database.
20Drop DatabaseDROP DATABASE enterprise_db;Deletes database instance.
21Check Replication StatusSHOW REPLICA STATUS\GInspects replica lag and sync.
22Reset ReplicaRESET REPLICA ALL;Clears replication configuration.
23Change Master ReplicaCHANGE REPLICATION SOURCE TO SOURCE_HOST='10.0.0.5';Sets replication upstream host.
24Create IndexALTER TABLE users ADD INDEX idx_email (email);Provisions new table index.
25Drop IndexALTER TABLE users DROP INDEX idx_email;Removes table index.
26Flush Query CacheRESET QUERY CACHE;Clears memory query cache.
27Show Master LogsSHOW BINARY LOGS;Lists active binary log files.
28Purge Binary LogsPURGE BINARY LOGS BEFORE '2026-06-01 00:00:00';Removes old binlog files.
29Set Global VariableSET GLOBAL max_connections = 500;Modifies runtime configuration.
30Check Disk StorageSELECT table_schema, SUM(data_length + index_length) FROM information_schema.tables GROUP BY table_schema;Calculates database storage sizes.
31Start Replica ThreadSTART REPLICA;Starts SQL and IO replication threads.
32Stop Replica ThreadSTOP REPLICA;Stops SQL and IO replication threads.
33Show Master StatusSHOW BINARY LOG STATUS;Inspects active binlog filename and offset position.
34Flush PrivilegesFLUSH PRIVILEGES;Reloads grant tables from mysql database.
35Flush LogsFLUSH LOGS;Flushes error log, binlog, and relay logs.
36Flush StatusFLUSH STATUS;Resets status variables to zero.
37Show Engine StatusSHOW ENGINE PERFORMANCE_SCHEMA STATUS;Inspects performance schema status.
38Show Open TablesSHOW OPEN TABLES WHERE In_use > 0;Lists tables currently locked by queries.
39Show Table StatusSHOW TABLE STATUS FROM enterprise_db;Pulls rows, data length, and engine types.
40Set Password ExpiryALTER USER 'app'@'localhost' PASSWORD EXPIRE INTERVAL 90 DAY;Enforces password rotation policy.
41Lock User AccountALTER USER 'app'@'localhost' ACCOUNT LOCK;Disables user authentication.
42Unlock User AccountALTER USER 'app'@'localhost' ACCOUNT UNLOCK;Enables user authentication.
43Show User GrantsSHOW GRANTS FOR 'app'@'localhost';Lists all assigned permissions for user.
44Revoke PrivilegesREVOKE ALL PRIVILEGES ON enterprise_db.* FROM 'app'@'localhost';Removes permissions from user.
45Create RoleCREATE ROLE 'app_read_only';Provisions role container in MySQL.
46Grant Role to UserGRANT 'app_read_only' TO 'app'@'localhost';Assigns role to user account.
47Set Default RoleSET DEFAULT ROLE ALL TO 'app'@'localhost';Activates roles on user login.
48Drop RoleDROP ROLE 'app_read_only';Removes role container.
49Show Binary Log EventsSHOW BINLOG EVENTS IN 'binlog.000001' LIMIT 10;Inspects specific transaction log events.
50Set GTID ModeSET GLOBAL gtid_mode = ON_PERMISSIVE;Enables Global Transaction Identifier mode.
51Check GTID ExecutedSELECT @@GLOBAL.gtid_executed;Inspects list of executed GTID transactions.
52Set InnoDB Buffer Pool SizeSET GLOBAL innodb_buffer_pool_size = 8589934592;Adjusts InnoDB buffer pool RAM size.
53Set InnoDB Log File SizeSET GLOBAL innodb_redo_log_capacity = 2147483648;Adjusts redo log capacity size.
54Check InnoDB Lock WaitsSELECT * FROM performance_schema.data_lock_waits;Inspects blocking transaction lock waits.
55Check InnoDB Data LocksSELECT * FROM performance_schema.data_locks;Inspects active row and table locks.
56Check InnoDB MetricsSELECT * FROM information_schema.INNODB_METRICS WHERE status='enabled';Pulls internal engine telemetry counters.
57Enable InnoDB MetricSET GLOBAL innodb_cmp_per_index_enabled = ON;Activates per-index compression metrics.
58Set Slow Query LogSET GLOBAL slow_query_log = 'ON';Activates slow query logging.
59Set Long Query TimeSET GLOBAL long_query_time = 2.0;Log queries taking longer than 2 seconds.
60Set Log Queries Not Using IndexesSET GLOBAL log_queries_not_using_indexes = ON;Logs queries missing index execution.
61Set Read Only ModeSET GLOBAL read_only = ON;Blocks writes for non-superusers.
62Set Super Read Only ModeSET GLOBAL super_read_only = ON;Blocks writes for all users including root.
63Disable Read Only ModeSET GLOBAL read_only = OFF;Enables database write operations.
64Check System VariablesSHOW VARIABLES LIKE 'max_allowed_packet';Inspects maximum packet size parameter.
65Set Max Connection TimeoutSET GLOBAL wait_timeout = 600;Closes non-interactive client connections after 10m.
66Create Partitioned TableCREATE TABLE logs (id INT, dt DATE) PARTITION BY RANGE (YEAR(dt)) (PARTITION p2026 VALUES LESS THAN (2027));Provisions range-partitioned table.
67Add Table PartitionALTER TABLE logs ADD PARTITION (PARTITION p2027 VALUES LESS THAN (2028));Appends new range partition to table.
68Drop Table PartitionALTER TABLE logs DROP PARTITION p2026;Purges partition data instantly.
69Truncate Table PartitionALTER TABLE logs TRUNCATE PARTITION p2027;Clears partition records.
70Check Partition InfoSELECT PARTITION_NAME, TABLE_ROWS FROM information_schema.PARTITIONS WHERE TABLE_NAME='logs';Inspects table partition sizes.
71Create Spatial IndexALTER TABLE locations ADD SPATIAL INDEX idx_geo (coords);Provisions R-Tree index for spatial data.
72Create Fulltext IndexALTER TABLE articles ADD FULLTEXT INDEX idx_ft (content);Provisions inverted index for text search.
73Run Fulltext QuerySELECT * FROM articles WHERE MATCH(content) AGAINST('database' IN BOOLEAN MODE);Executes fulltext search query.
74Check Table Column TypesDESCRIBE users;Outputs table column definitions.
75Rename TableRENAME TABLE users TO app_users;Renames table relation in database.
76Add Foreign KeyALTER TABLE orders ADD CONSTRAINT fk_user FOREIGN KEY (user_id) REFERENCES users(id);Enforces foreign key constraint.
77Drop Foreign KeyALTER TABLE orders DROP FOREIGN KEY fk_user;Removes foreign key constraint.
78Create Event SchedulerCREATE EVENT purge_logs ON SCHEDULE EVERY 1 DAY DO DELETE FROM logs WHERE dt < NOW() - INTERVAL 30 DAY;Provisions automated scheduled event.
79Enable Event SchedulerSET GLOBAL event_scheduler = ON;Activates background event execution daemon.
80Show Scheduled EventsSHOW EVENTS FROM enterprise_db;Lists all configured scheduled events.
81Drop Scheduled EventDROP EVENT IF EXISTS purge_logs;Removes scheduled background event.
82Create Stored ProcedureDELIMITER // CREATE PROCEDURE CleanUsers() BEGIN DELETE FROM users WHERE active=0; END // DELIMITER ;Provisions stored procedure routine.
83Execute Stored ProcedureCALL CleanUsers();Runs stored procedure routine.
84Drop Stored ProcedureDROP PROCEDURE IF EXISTS CleanUsers;Deletes stored procedure.
85Create Stored FunctionCREATE FUNCTION AddTax(p DECIMAL(10,2)) RETURNS DECIMAL(10,2) DETERMINISTIC RETURN p * 1.08;Provisions scalar user function.
86Drop Stored FunctionDROP FUNCTION IF EXISTS AddTax;Deletes stored function.
87Check Performance Schema MemorySELECT * FROM performance_schema.memory_summary_global_by_event_name ORDER BY CURRENT_NUMBER_OF_BYTES_USED DESC LIMIT 10;Analyzes memory allocation across events.
88Check Statement Execution StatsSELECT * FROM performance_schema.events_statements_summary_by_digest ORDER BY SUM_TIMER_WAIT DESC LIMIT 10;Finds most resource-intensive SQL queries.
89Check Table I/O StatsSELECT * FROM performance_schema.table_io_waits_summary_by_table ORDER BY SUM_TIMER_WAIT DESC LIMIT 10;Measures physical I/O latency across tables.
90Check File I/O StatsSELECT * FROM performance_schema.file_summary_by_event_name ORDER BY SUM_TIMER_WAIT DESC LIMIT 10;Measures physical file I/O latency.
91Create User SavepointSAVEPOINT sp1;Sets savepoint within active transaction.
92Rollback to SavepointROLLBACK TO SAVEPOINT sp1;Reverts transaction state back to savepoint.
93Release SavepointRELEASE SAVEPOINT sp1;Removes savepoint reference.
94Check Transaction Isolation LevelSELECT @@transaction_isolation;Inspects active session isolation mode.
95Set Transaction Isolation LevelSET SESSION TRANSACTION ISOLATION LEVEL READ COMMITTED;Configures read committed transaction isolation.
96Check Max Allowed PacketSELECT @@max_allowed_packet;Inspects max network payload threshold.
97Check Character SetsSHOW CHARACTER SET;Outputs supported database character sets.
98Check CollationsSHOW COLLATION WHERE Charset='utf8mb4';Lists supported utf8mb4 collation orderings.
99Verify Engine VersionSELECT VERSION();Outputs running MySQL engine version.
100Shutdown Server via SQLSHUTDOWN;Triggers graceful server shutdown via SQL command.
101Check InnoDB Buffer Pool Hit RateSELECT (1 - (SELECT VARIABLE_VALUE FROM performance_schema.global_status WHERE VARIABLE_NAME = 'Innodb_buffer_pool_reads') / (SELECT VARIABLE_VALUE FROM performance_schema.global_status WHERE VARIABLE_NAME = 'Innodb_buffer_pool_read_requests')) * 100 AS Hit_Rate;Calculates InnoDB buffer pool hit ratio percentage.
102Check InnoDB Redo Log Wait RateSHOW GLOBAL STATUS LIKE 'Innodb_log_waits';Measures redo log write waits.
103Check InnoDB Lock Waiting TimeSHOW GLOBAL STATUS LIKE 'Innodb_row_lock_time_avg';Measures average row lock wait latency.
104Check Table Cache StatusSHOW GLOBAL STATUS LIKE 'Open_tables';Inspects open table handles count.
105Set Table Open CacheSET GLOBAL table_open_cache = 4000;Increases open table handle capacity.
106Check Thread Cache SizeSHOW GLOBAL STATUS LIKE 'Threads_cached';Inspects cached connection threads count.
107Set Thread Cache SizeSET GLOBAL thread_cache_size = 64;Configures connection thread recycling capacity.
108Check Max Execution TimeSET SESSION max_execution_time = 5000;Caps SELECT statement execution time to 5s.
109Check Connection Limits per UserALTER USER 'app'@'localhost' WITH MAX_CONNECTIONS_PER_HOUR 1000;Throttles user connections per hour.
110Check Account Resource LimitsALTER USER 'app'@'localhost' WITH MAX_QUERIES_PER_HOUR 50000;Throttles total queries per hour.
111Set Password LifetimeALTER USER 'app'@'localhost' PASSWORD EXPIRE INTERVAL 180 DAY;Configures 180-day password expiration.
112Set Password HistoryALTER USER 'app'@'localhost' PASSWORD HISTORY 5;Prevents reusing last 5 passwords.
113Check Binary Log ExpirySHOW VARIABLES LIKE 'binlog_expire_logs_seconds';Inspects automatic binlog purge window.
114Set Binary Log ExpirySET GLOBAL binlog_expire_logs_seconds = 604800;Configures 7-day automatic binlog purge.
115Check Binlog FormatSHOW VARIABLES LIKE 'binlog_format';Inspects replication log format (ROW, STATEMENT, MIXED).
116Set Binlog Format RowSET GLOBAL binlog_format = 'ROW';Enforces row-based replication logging.
117Check Group Replication StatusSELECT * FROM performance_schema.replication_group_members;Inspects active Galera/Group Replication nodes.
118Check Group Replication Member StatsSELECT * FROM performance_schema.replication_group_member_stats;Monitors queue lag across group nodes.
119Check Performance Schema Stage ProgressSELECT * FROM performance_schema.events_stages_current;Monitors progress of ALTER TABLE / OPTIMIZE operations.
120Check Index Usage DigestSELECT * FROM performance_schema.schema_index_statistics;Monitors read/write I/O per index.
121Check Unused Indexes PSSELECT * FROM sys.schema_unused_indexes;Identifies candidate indexes for removal via sys schema.
122Check Duplicate Indexes PSSELECT * FROM sys.schema_redundant_indexes;Identifies redundant overlapping indexes via sys schema.
123Check Table Auto Increment ValueSELECT AUTO_INCREMENT FROM information_schema.tables WHERE table_name='users';Inspects current auto-increment counter.
124Set Table Auto Increment ValueALTER TABLE users AUTO_INCREMENT = 10000;Resets auto-increment counter.
125Check InnoDB Table CompressionALTER TABLE users ROW_FORMAT=COMPRESSED KEY_BLOCK_SIZE=8;Applies page-level compression.
126Check Query Rewrite PluginSHOW PLUGINS;Inspects installed server plugins.
127Install PluginINSTALL PLUGIN audit_log SONAME 'audit_log.so';Installs Enterprise Audit plugin.
128Uninstall PluginUNINSTALL PLUGIN audit_log;Removes server plugin.
129Check Engine Version DetailsSELECT @@version_comment, @@version_compile_os;Outputs OS build environment and comment.
130Verify MySQL UptimeSHOW GLOBAL STATUS LIKE 'Uptime';Displays server uptime seconds.
131Check InnoDB Buffer Pool InstancesSHOW VARIABLES LIKE 'innodb_buffer_pool_instances';Inspects buffer pool shard count.
132Check InnoDB Page SizeSHOW VARIABLES LIKE 'innodb_page_size';Inspects physical InnoDB page size in bytes.
133Check InnoDB Flush Log At Trx CommitSHOW VARIABLES LIKE 'innodb_flush_log_at_trx_commit';Inspects ACID redo log flush safety (0, 1, 2).
134Set InnoDB Flush Log At Trx CommitSET GLOBAL innodb_flush_log_at_trx_commit = 1;Enforces strict ACID redo log disk flush on commit.
135Check InnoDB Doublewrite BufferSHOW VARIABLES LIKE 'innodb_doublewrite';Inspects doublewrite buffer safety mode.
136Check InnoDB File Per TableSHOW VARIABLES LIKE 'innodb_file_per_table';Verifies whether each table gets its own .ibd file.
137Check InnoDB Adaptive Hash IndexSHOW VARIABLES LIKE 'innodb_adaptive_hash_index';Inspects adaptive hash index state.
138Disable InnoDB Adaptive Hash IndexSET GLOBAL innodb_adaptive_hash_index = OFF;Disables adaptive hash index to reduce mutex contention.
139Enable InnoDB Adaptive Hash IndexSET GLOBAL innodb_adaptive_hash_index = ON;Activates adaptive hash index.
140Check InnoDB Read I/O ThreadsSHOW VARIABLES LIKE 'innodb_read_io_threads';Inspects background disk read threads.
141Check InnoDB Write I/O ThreadsSHOW VARIABLES LIKE 'innodb_write_io_threads';Inspects background disk write threads.
142Check InnoDB Max Dirty Pages PctSHOW VARIABLES LIKE 'innodb_max_dirty_pages_pct';Inspects max dirty page ratio in buffer pool.
143Check InnoDB Flush MethodSHOW VARIABLES LIKE 'innodb_flush_method';Inspects OS file flush method (O_DIRECT).
144Check InnoDB IO CapacitySHOW VARIABLES LIKE 'innodb_io_capacity';Inspects base I/O operations per second allocation.
145Set InnoDB IO CapacitySET GLOBAL innodb_io_capacity = 2000;Increases base I/O capacity for SSD storage.
146Check InnoDB IO Capacity MaxSHOW VARIABLES LIKE 'innodb_io_capacity_max';Inspects peak I/O capacity for background flushing.
147Set InnoDB IO Capacity MaxSET GLOBAL innodb_io_capacity_max = 4000;Increases max I/O capacity for SSD storage.
148Check InnoDB Lock Wait TimeoutSHOW VARIABLES LIKE 'innodb_lock_wait_timeout';Inspects max seconds query waits for row lock.
149Set InnoDB Lock Wait TimeoutSET GLOBAL innodb_lock_wait_timeout = 10;Caps row lock wait duration to 10 seconds.
150Check InnoDB Print Deadlock LogsSHOW VARIABLES LIKE 'innodb_print_all_deadlocks';Inspects if all deadlocks are written to error log.
151Enable InnoDB Print Deadlock LogsSET GLOBAL innodb_print_all_deadlocks = ON;Logs all transaction deadlocks to error log.
152Check Binlog Cache SizeSHOW VARIABLES LIKE 'binlog_cache_size';Inspects transaction binary log buffer RAM size.
153Check Sync BinlogSHOW VARIABLES LIKE 'sync_binlog';Inspects binary log disk sync frequency (0, 1).
154Set Sync BinlogSET GLOBAL sync_binlog = 1;Enforces disk sync for binary log on every transaction.
155Check Relay Log InfoSHOW RELAYLOG EVENTS IN 'relaylog.000001' LIMIT 10;Inspects contents of replication relay log.
156Check Replica Parallel WorkersSHOW VARIABLES LIKE 'replica_parallel_workers';Inspects parallel replication applier threads.
157Set Replica Parallel WorkersSET GLOBAL replica_parallel_workers = 8;Enables 8 parallel worker threads for replication.
158Check Replica Parallel TypeSHOW VARIABLES LIKE 'replica_parallel_type';Inspects parallel replication dependency algorithm (LOGICAL_CLOCK).
159Set Replica Parallel TypeSET GLOBAL replica_parallel_type = 'LOGICAL_CLOCK';Enforces logical clock dependency parallel replication.
160Check Replica Preserve Commit OrderSHOW VARIABLES LIKE 'replica_preserve_commit_order';Ensures replica commits transactions in master order.
161Set Replica Preserve Commit OrderSET GLOBAL replica_preserve_commit_order = ON;Enforces commit order preservation on replica.
162Check Master Info FileSHOW VARIABLES LIKE 'master_info_repository';Inspects replication metadata storage (TABLE vs FILE).
163Check Relay Log Info FileSHOW VARIABLES LIKE 'relay_log_info_repository';Inspects relay log metadata storage (TABLE vs FILE).
164Check Character Set DatabaseSHOW VARIABLES LIKE 'character_set_database';Inspects active database default character set.
165Check Collation DatabaseSHOW VARIABLES LIKE 'collation_database';Inspects active database default collation.
166Check Character Set ServerSHOW VARIABLES LIKE 'character_set_server';Inspects server-wide default character set.
167Check Collation ServerSHOW VARIABLES LIKE 'collation_server';Inspects server-wide default collation.
168Set Character Set ClientSET NAMES 'utf8mb4';Configures client connection character set and collation.
169Check Auto Increment IncrementSHOW VARIABLES LIKE 'auto_increment_increment';Inspects auto-increment step value for multi-master.
170Check Auto Increment OffsetSHOW VARIABLES LIKE 'auto_increment_offset';Inspects auto-increment starting offset for multi-master.
171Set Auto Increment StepSET GLOBAL auto_increment_increment = 2;Configures 2-step increment for dual-master replication.
172Set Auto Increment OffsetSET GLOBAL auto_increment_offset = 1;Configures offset 1 for master 1.
173Check Max Allowed PacketSHOW VARIABLES LIKE 'max_allowed_packet';Inspects maximum allowed network packet payload size.
174Set Max Allowed PacketSET GLOBAL max_allowed_packet = 1073741824;Increases max packet payload size to 1GB.
175Check Interactive TimeoutSHOW VARIABLES LIKE 'interactive_timeout';Inspects timeout for interactive client connections.
176Check Wait TimeoutSHOW VARIABLES LIKE 'wait_timeout';Inspects timeout for non-interactive client connections.
177Check Max Heap Table SizeSHOW VARIABLES LIKE 'max_heap_table_size';Inspects max size for MEMORY engine tables.
178Check Tmp Table SizeSHOW VARIABLES LIKE 'tmp_table_size';Inspects max size for in-memory internal temp tables.
179Check Created Tmp Disk TablesSHOW GLOBAL STATUS LIKE 'Created_tmp_disk_tables';Monitors temp tables spilling to disk.
180Check Created Tmp TablesSHOW GLOBAL STATUS LIKE 'Created_tmp_tables';Monitors total internal temp tables created.
181Check Select Scan CountSHOW GLOBAL STATUS LIKE 'Select_scan';Monitors full table scan execution count.
182Check Select Full Join CountSHOW GLOBAL STATUS LIKE 'Select_full_join';Monitors joins missing index keys.
183Check Slow Queries CountSHOW GLOBAL STATUS LIKE 'Slow_queries';Monitors total slow queries executed.
184Check Com Select CountSHOW GLOBAL STATUS LIKE 'Com_select';Monitors SELECT statements executed.
185Check Com Insert CountSHOW GLOBAL STATUS LIKE 'Com_insert';Monitors INSERT statements executed.
186Check Com Update CountSHOW GLOBAL STATUS LIKE 'Com_update';Monitors UPDATE statements executed.
187Check Com Delete CountSHOW GLOBAL STATUS LIKE 'Com_delete';Monitors DELETE statements executed.
188Check Connections CountSHOW GLOBAL STATUS LIKE 'Connections';Monitors total connection attempts since boot.
189Check Aborted ConnectsSHOW GLOBAL STATUS LIKE 'Aborted_connects';Monitors failed client connection attempts.
190Check Aborted ClientsSHOW GLOBAL STATUS LIKE 'Aborted_clients';Monitors client connections dropped improperly.
191Check Query PingSELECT 1;Simple database ping test returning 1.
192Check Current User AccountSELECT CURRENT_USER();Returns current execution user and host.
193Check Current Database NameSELECT DATABASE();Returns active selected database name.
194Check System TimestampSELECT NOW();Returns current server date and time.
195Check System UTC TimestampSELECT UTC_TIMESTAMP();Returns current UTC date and time.
196Check System Connection IDSELECT CONNECTION_ID();Returns current connection thread ID.
197Check MySQL Version DetailsSELECT VERSION();Outputs MySQL server build and patch version.
198Check Server Uptime StatusSHOW GLOBAL STATUS LIKE 'Uptime';Displays server uptime in seconds.
199Check Error Log LocationSHOW VARIABLES LIKE 'log_error';Inspects file path of server error log.
200Check Host Name VariableSHOW VARIABLES LIKE 'hostname';Outputs host server hostname.

SQL Server T-SQL OLTP

Technical Architecture & Overview

Microsoft SQL Server (T-SQL) is a relational enterprise database platform featuring advanced memory-optimized tables (In-Memory OLTP / Hekaton), Columnstore indexes, AlwaysOn Availability Groups, transparent data encryption (TDE), and deep integration with Windows/Linux enterprise ecosystems.

Primary Use Cases: Enterprise ERP/CRM transaction engines, high-concurrency financial systems, data warehousing, and mission-critical line-of-business applications.

Core Components: Relational Engine, Storage Engine, Buffer Pool, Transaction Log (WAL), TempDB, Query Store, and AlwaysOn High Availability AGs.

Exhaustive 200-command administrative, diagnostic, and operational matrix for SQL Server T-SQL.

#Operation / ActionT-SQL / PowerShell SyntaxDescription
1Check Active SessionsSELECT * FROM sys.dm_exec_sessions WHERE is_user_process = 1;Inspects client connections.
2Check Active QueriesSELECT * FROM sys.dm_exec_requests;Views running queries.
3Kill Process SessionKILL 54;Terminates blocking SPID.
4Backup DatabaseBACKUP DATABASE EnterpriseDB TO DISK = 'C:\Backup\db.bak' WITH COMPRESSION;Generates compressed SQL backup.
5Restore DatabaseRESTORE DATABASE EnterpriseDB FROM DISK = 'C:\Backup\db.bak';Restores database backup.
6Rebuild IndexesALTER INDEX ALL ON Users REBUILD WITH (ONLINE = ON);Rebuilds indexes without downtime.
7Update StatisticsUPDATE STATISTICS Users WITH FULLSCAN;Refreshes query optimizer stats.
8Check Database IntegrityDBCC CHECKDB (EnterpriseDB);Scans for physical corruption.
9Shrink Database LogDBCC SHRINKFILE (EnterpriseDB_Log, 100);Truncates transaction log size.
10Create LoginCREATE LOGIN AppUser WITH PASSWORD = 'SecurePassword123!';Provisions server-level login.
11Create Database UserCREATE USER AppUser FOR LOGIN AppUser;Maps login to database user.
12Grant Role PermissionsALTER ROLE db_datareader ADD MEMBER AppUser;Grants read-only access rights.
13Check Index FragmentationSELECT * FROM sys.dm_db_index_physical_stats(DB_ID(), NULL, NULL, NULL, 'LIMITED');Evaluates index fragmentation levels.
14View Blocking ChainsSELECT * FROM sys.dm_os_waiting_tasks WHERE blocking_session_id IS NOT NULL;Identifies blocking locks.
15Get Database SizesSELECT name, size * 8 / 1024 AS SizeMB FROM sys.master_files;Calculates file storage use.
16Clear Plan CacheDBCC FREEPROCCACHE;Flushes compiled query plans.
17Clear Buffer PoolDBCC DROPCLEANBUFFERS;Flushes clean buffers from memory.
18Enable Extended EventsALTER EVENT SESSION [SystemDiagnostics] ON SERVER STATE = START;Activates diagnostic telemetry.
19Create DatabaseCREATE DATABASE EnterpriseDB;Provisions new database.
20Drop DatabaseDROP DATABASE EnterpriseDB;Deletes database instance.
21Check AlwaysOn StatusSELECT * FROM sys.dm_hadr_availability_replica_states;Inspects high availability sync.
22Failover Availability GroupALTER AVAILABILITY GROUP EnterpriseAG FAILOVER;Triggers manual AG failover.
23Set Recovery ModelALTER DATABASE EnterpriseDB SET RECOVERY FULL;Configures transaction log recovery.
24Create SchemaCREATE AUTHORIZATION ON SCHEMA::dbo TO AppUser;Modifies schema ownership.
25Get Execution StatsSELECT TOP 10 text, execution_count FROM sys.dm_exec_query_stats qs CROSS APPLY sys.dm_exec_sql_text(qs.sql_handle);Finds most expensive queries.
26Configure Max Server MemoryEXEC sp_configure 'max server memory (MB)', 16384; RECONFIGURE;Limits buffer pool RAM usage.
27Check Agent JobsEXEC msdb.dbo.sp_help_job;Lists scheduled SQL Agent jobs.
28Run Agent JobEXEC msdb.dbo.sp_start_job N'NightlyBackup';Manually triggers job execution.
29View Deadlock GraphsSELECT xed.value('(event/@name)[1]', 'varchar(150)') FROM sys.fn_xe_file_target_read_file('system_health*.xel', null, null, null);Inspects historical deadlock logs.
30Check Service Statuspowershell "Get-Service MSSQLSERVER"Verifies Windows service state.
31Reorganize IndexALTER INDEX ALL ON Users REORGANIZE;Defragments index pages in-place.
32Set Page Verify OptionALTER DATABASE EnterpriseDB SET PAGE_VERIFY CHECKSUM;Enforces page checksum validation.
33Set Auto Shrink OffALTER DATABASE EnterpriseDB SET AUTO_SHRINK OFF;Disables auto-shrink to prevent fragmentation.
34Set Auto Create StatsALTER DATABASE EnterpriseDB SET AUTO_CREATE_STATISTICS ON;Enables automatic optimizer stats creation.
35Set Auto Update StatsALTER DATABASE EnterpriseDB SET AUTO_UPDATE_STATISTICS ON;Enables automatic optimizer stats updates.
36Set Auto Update Stats AsyncALTER DATABASE EnterpriseDB SET AUTO_UPDATE_STATISTICS_ASYNC ON;Asynchronously updates optimizer statistics.
37Set Parameter Sniffing OffALTER DATABASE SCOPED CONFIGURATION SET PARAMETER_SNIFFING = OFF;Disables parameter sniffing for query plans.
38Set Max Degree of ParallelismALTER DATABASE SCOPED CONFIGURATION SET MAXDOP = 4;Caps query execution parallel DOP.
39Set Query Store OnALTER DATABASE EnterpriseDB SET QUERY_STORE = ON;Enables Query Store performance recorder.
40Clear Query StoreALTER DATABASE EnterpriseDB SET QUERY_STORE CLEAR;Purges recorded Query Store telemetry.
41Check Query Store Runtime StatsSELECT * FROM sys.query_store_runtime_stats;Inspects execution duration and CPU usage.
42Check Query Store PlansSELECT * FROM sys.query_store_plan;Inspects compiled execution plans in Query Store.
43Force Query PlanEXEC sp_query_store_force_plan @query_id = 10, @plan_id = 5;Forces Query Store to use specific plan.
44Unforce Query PlanEXEC sp_query_store_unforce_plan @query_id = 10, @plan_id = 5;Unforces query plan override.
45Take Differential BackupBACKUP DATABASE EnterpriseDB TO DISK = 'C:\Backup\db_diff.bak' WITH DIFFERENTIAL;Backs up changes since full backup.
46Take Log BackupBACKUP LOG EnterpriseDB TO DISK = 'C:\Backup\log.trn';Backs up active transaction log.
47Restore Log BackupRESTORE LOG EnterpriseDB FROM DISK = 'C:\Backup\log.trn' WITH NORECOVERY;Restores transaction log chain.
48Restore Final LogRESTORE LOG EnterpriseDB FROM DISK = 'C:\Backup\log_final.trn' WITH RECOVERY;Finalizes database recovery sequence.
49Check Backup HistorySELECT * FROM msdb.dbo.backupset;Inspects historical database backup logs.
50Check Restore HistorySELECT * FROM msdb.dbo.restorehistory;Inspects historical database restore logs.
51Check Suspect PagesSELECT * FROM msdb.dbo.suspect_pages;Inspects pages flagged for I/O errors.
52Check DBCC InputbufferDBCC INPUTBUFFER(54);Shows last SQL statement sent by SPID.
53Check DBCC OPENTRANDBCC OPENTRAN(EnterpriseDB);Identifies oldest active open transaction.
54Check DBCC SQLPERFDBCC SQLPERF(LOGSPACE);Monitors transaction log space usage.
55Check DBCC USEROPTIONSDBCC USEROPTIONS;Inspects active session SET configuration options.
56Grant Server RoleALTER SERVER ROLE sysadmin ADD MEMBER AppUser;Grants sysadmin server privileges.
57Deny Table SelectDENY SELECT ON Users TO AppUser;Explicitly blocks read access on table.
58Revoke PermissionREVOKE SELECT ON Users FROM AppUser;Removes granted or denied permission.
59Create Application RoleCREATE APPLICATION ROLE AppRole WITH PASSWORD = 'RolePass123!';Provisions application role.
60Create Server AuditCREATE SERVER AUDIT AppAudit TO FILE (FILEPATH = 'C:\Audit\');Provisions server audit specification.
61Enable Server AuditALTER SERVER AUDIT AppAudit WITH (STATE = ON);Activates server audit logging.
62Create Database Audit SpecCREATE DATABASE AUDIT SPECIFICATION DbAudit SPECIFICATION FOR SERVER AUDIT AppAudit ADD (SELECT ON Users BY PUBLIC);Audits queries on table.
63Enable Database Audit SpecALTER DATABASE AUDIT SPECIFICATION DbAudit WITH (STATE = ON);Activates database audit spec.
64Create Column Encryption KeyCREATE COLUMN ENCRYPTION KEY CEK1 WITH VALUES (COLUMN_MASTER_KEY = CMK1, ALGORITHM = 'RSA_OAEP');Provisions Always Encrypted key.
65Create Columnstore IndexCREATE CLUSTERED COLUMNSTORE INDEX cci_facts ON FactSales;Provisions high-speed analytics Columnstore.
66Create Nonclustered ColumnstoreCREATE NONCLUSTERED COLUMNSTORE INDEX ncci_sales ON FactSales(Amount, DateKey);Provisions secondary Columnstore index.
67Reorganize Columnstore IndexALTER INDEX cci_facts ON FactSales REORGANIZE WITH (COMPRESS_ALL_ROW_GROUPS = ON);Compacts Rowgroups in Columnstore.
68Create Partition SchemeCREATE PARTITION SCHEME ps_date AS PARTITION pf_date TO (fg1, fg2, fg3);Maps partitions to filegroups.
69Create Partition FunctionCREATE PARTITION FUNCTION pf_date (datetime2) AS RANGE RIGHT FOR VALUES ('2026-01-01', '2027-01-01');Provisions partition boundaries.
70Split Partition RangeALTER PARTITION FUNCTION pf_date() SPLIT RANGE ('2026-07-01');Splits existing partition range.
71Merge Partition RangeALTER PARTITION FUNCTION pf_date() MERGE RANGE ('2026-07-01');Merges adjacent partition ranges.
72Switch Partition TableALTER TABLE Sales SWITCH PARTITION 1 TO SalesArchive PARTITION 1;Instantly moves partition data.
73Create FilegroupALTER DATABASE EnterpriseDB ADD FILEGROUP FG_Data2026;Provisions secondary filegroup.
74Add File to FilegroupALTER DATABASE EnterpriseDB ADD FILE (NAME = N'Data2026', FILENAME = N'C:\Data\Data2026.ndf') TO FILEGROUP FG_Data2026;Adds storage file to filegroup.
75Set Default FilegroupALTER DATABASE EnterpriseDB MODIFY FILEGROUP FG_Data2026 DEFAULT;Sets primary target filegroup.
76Check TempDB ContentionSELECT * FROM sys.dm_os_waiting_tasks WHERE page_workspace_clone = 1 OR resource_description LIKE '2:%';Monitors TempDB allocation bottleneck.
77Check TempDB UsageSELECT * FROM sys.dm_db_file_space_usage;Monitors TempDB file utilization.
78Set Containment TypeALTER DATABASE EnterpriseDB SET CONTAINMENT = PARTIAL;Configures contained database model.
79Set Change Data Capture OnEXEC sys.sp_cdc_enable_db;Activates CDC on database level.
80Enable CDC on TableEXEC sys.sp_cdc_enable_table @source_schema = N'dbo', @source_name = N'Users', @role_name = NULL;Tracks table changes via CDC.
81Disable CDC on TableEXEC sys.sp_cdc_disable_table @source_schema = N'dbo', @source_name = N'Users', @capture_instance = N'all';Stops CDC tracking on table.
82Check CDC ErrorsSELECT * FROM sys.dm_cdc_errors;Inspects CDC capture and cleanup job errors.
83Set Change Tracking OnALTER DATABASE EnterpriseDB SET CHANGE_TRACKING = ON (CHANGE_RETENTION = 2 DAYS, AUTO_CLEANUP = ON);Activates Change Tracking.
84Enable Change Tracking TableALTER TABLE Users ENABLE CHANGE_TRACKING;Enforces change tracking on table.
85Get Change Tracking ChangesSELECT * FROM CHANGETABLE(CHANGES Users, 0) AS CT;Pulls table mutation history.
86Set Snapshot IsolationALTER DATABASE EnterpriseDB SET ALLOW_SNAPSHOT_ISOLATION ON;Enforces row-versioning snapshot isolation.
87Set Read Committed SnapshotALTER DATABASE EnterpriseDB SET READ_COMMITTED_SNAPSHOT ON;Eliminates read locks using RCSI.
88Check Wait StatisticsSELECT * FROM sys.dm_os_wait_stats ORDER BY wait_time_ms DESC;Pulls server-wide wait type statistics.
89Clear Wait StatisticsDBCC SQLPERF('sys.dm_os_wait_stats', CLEAR);Resets wait statistics memory counters.
90Check Memory ClerksSELECT type, SUM(pages_kb) FROM sys.dm_os_memory_clerks GROUP BY type ORDER BY SUM(pages_kb) DESC;Analyzes memory allocation breakdown.
91Check Ring Buffer EventsSELECT * FROM sys.dm_os_ring_buffers;Inspects internal memory and exception events.
92Check LatchesSELECT * FROM sys.dm_os_latch_stats ORDER BY wait_time_ms DESC;Analyzes thread latch contention.
93Check SpinlocksSELECT * FROM sys.dm_os_spinlock_stats ORDER BY spins DESC;Analyzes CPU spinlock contention.
94Check Buffer DescriptorsSELECT count(*)*8/1024 AS MB, DB_NAME(database_id) FROM sys.dm_os_buffer_descriptors GROUP BY database_id;Inspects RAM cache usage per DB.
95Check Missing Index DMVSELECT * FROM sys.dm_db_missing_index_details;Finds recommended indexes from optimizer.
96Check Unused Index DMVSELECT * FROM sys.dm_db_index_usage_stats WHERE user_scans = 0 AND user_seeks = 0;Finds indexes with zero query usage.
97Create Extended Event SessionCREATE EVENT SESSION [XE_SlowQueries] ON SERVER ADD EVENT sqlserver.sql_statement_completed(WHERE duration > 1000000);Provisions XE tracing for slow queries.
98Drop Extended Event SessionDROP EVENT SESSION [XE_SlowQueries] ON SERVER;Removes Extended Event session.
99Verify Product VersionSELECT @@VERSION;Outputs running SQL Server build and patch version.
100Set Database OfflineALTER DATABASE EnterpriseDB SET OFFLINE WITH ROLLBACK IMMEDIATE;Forces database offline instantly.
101Check TempDB Allocation ContentionSELECT * FROM sys.dm_os_waiting_tasks WHERE resource_description LIKE '2:%';Identifies PFS/GAM page contention in TempDB.
102Add TempDB Data FilesALTER DATABASE tempdb ADD FILE (NAME = N'tempdev2', FILENAME = N'C:\TempDB\tempdev2.ndf', SIZE = 1024MB);Adds data file to balance TempDB I/O.
103Check Memory Grant WarningsSELECT * FROM sys.dm_exec_query_memory_grants WHERE grant_time IS NULL;Identifies queries waiting for memory grants.
104Check Resource Governor PoolsSELECT * FROM sys.dm_resource_governor_resource_pools;Monitors CPU and RAM allocation across pools.
105Create Resource Governor PoolCREATE RESOURCE POOL BatchPool WITH (MAX_CPU_PERCENT = 50, MAX_MEMORY_PERCENT = 30);Caps batch workload CPU/RAM.
106Create Resource Governor GroupCREATE WORKLOAD GROUP BatchGroup USING BatchPool;Maps workload group to resource pool.
107Alter Resource Governor ReconfigureALTER RESOURCE GOVERNOR RECONFIGURE;Applies Resource Governor configuration changes.
108Check In-Memory OLTP TablesSELECT * FROM sys.memory_optimized_tables_internal_attributes;Inspects Hekaton in-memory tables.
109Create Memory Optimized TableCREATE TABLE dbo.FastCart (id INT NOT NULL PRIMARY KEY NONCLUSTERED HASH WITH (BUCKET_COUNT = 100000)) WITH (MEMORY_OPTIMIZED = ON, DURABILITY = SCHEMA_AND_DATA);Provisions Hekaton in-memory table.
110Check Columnstore Tuple UnitsSELECT * FROM sys.dm_db_column_store_row_group_physical_stats;Inspects Columnstore compressed rowgroups.
111Check AlwaysOn Listener StatusSELECT * FROM sys.availability_group_listeners;Inspects AG network listener IP and port.
112Check AlwaysOn Sync LagSELECT replica_server_name, log_send_queue_size, log_send_rate FROM sys.dm_hadr_database_replica_states;Monitors log send queue lag in AG.
113Set AlwaysOn Sync ModeALTER AVAILABILITY GROUP EnterpriseAG MODIFY REPLICA ON 'NODE2' WITH (AVAILABILITY_MODE = SYNCHRONOUS_COMMIT);Configures zero-data-loss sync mode.
114Check Backup VerificationRESTORE VERIFYONLY FROM DISK = 'C:\Backup\db.bak';Validates backup tape/disk file integrity.
115Check Page Verify ErrorSELECT DB_NAME(database_id), file_id, page_id, error_type FROM sys.dm_db_mirroring_auto_page_repair;Inspects automatic page repair logs.
116Check File Auto Growth EventsSELECT * FROM sys.fn_trace_gettable('C:\Trace\log.trc', default) WHERE EventClass IN (92, 93);Monitors file auto-expansion events.
117Set File Autogrowth In MBALTER DATABASE EnterpriseDB MODIFY FILE (NAME = N'EnterpriseDB', FILEGROWTH = 512MB);Enforces fixed MB autogrowth to prevent fragmentation.
118Check Database Encryption TDESELECT db_name(database_id), encryption_state FROM sys.dm_database_encryption_keys;Inspects TDE transparent encryption state.
119Enable Database EncryptionALTER DATABASE EnterpriseDB SET ENCRYPTION ON;Activates TDE background encryption.
120Check Extended Events SessionsSELECT * FROM sys.dm_xe_sessions;Lists active running Extended Event sessions.
121Stop Extended Events SessionALTER EVENT SESSION [SystemDiagnostics] ON SERVER STATE = STOP;Halts Extended Event session.
122Check Partition Ranges InfoSELECT * FROM sys.partition_range_values;Inspects boundary values of partition functions.
123Check Table Partition DistributionSELECT p.partition_number, p.rows FROM sys.partitions p WHERE p.object_id = OBJECT_ID('Sales');Inspects row count per partition.
124Check Database Compatibility LevelSELECT name, compatibility_level FROM sys.databases WHERE name = 'EnterpriseDB';Inspects T-SQL compatibility version.
125Set Database Compatibility LevelALTER DATABASE EnterpriseDB SET COMPATIBILITY_LEVEL = 160;Upgrades compatibility to SQL Server 2022.
126Check Server CollationSELECT SERVERPROPERTY('Collation');Outputs default server collation.
127Check Database User PermissionsSELECT * FROM sys.database_permissions WHERE grantee_principal_id = USER_ID('AppUser');Lists explicit permissions assigned to user.
128Check Server Role MembersSELECT * FROM sys.server_role_members;Lists accounts assigned to server admin roles.
129Check System Uptime SQLSELECT sqlserver_start_time FROM sys.dm_os_sys_info;Outputs exact SQL Server boot timestamp.
130Verify SQL Version StringSELECT @@VERSION;Outputs SQL Server edition and CU patch build string.
131Check Database StatusSELECT name, state_desc FROM sys.databases;Inspects database ONLINE/OFFLINE state.
132Check Database Recovery StateSELECT name, recovery_model_desc FROM sys.databases;Inspects recovery model (FULL, SIMPLE, BULK_LOGGED).
133Check Database Log Reuse WaitSELECT name, log_reuse_wait_desc FROM sys.databases;Identifies reason preventing log truncation.
134Check Database File AutogrowthSELECT name, growth, is_percent_growth FROM sys.master_files;Inspects autogrowth settings for files.
135Check Database File Space FreeSELECT name, size*8/1024 AS SizeMB, FILEPROPERTY(name, 'SpaceUsed')*8/1024 AS UsedMB FROM sys.database_files;Calculates free space inside data files.
136Check Database VLF CountSELECT name, COUNT(l.database_id) AS VLFCount FROM sys.databases d CROSS APPLY sys.dm_db_log_info(d.database_id) l GROUP BY name;Monitors Virtual Log File count to prevent degradation.
137Check TempDB Allocation PagesSELECT * FROM sys.dm_db_page_info(2, 1, 1, 'DETAILED');Inspects TempDB GAM page header details.
138Check Wait Stats Top 10SELECT TOP 10 wait_type, waiting_tasks_count, wait_time_ms FROM sys.dm_os_wait_stats WHERE wait_type NOT LIKE '%SLEEP%' ORDER BY wait_time_ms DESC;Lists top blocking wait types.
139Check Memory Grants PendingSELECT * FROM sys.dm_exec_query_memory_grants WHERE grant_time IS NULL;Lists queries queued waiting for RAM grant.
140Check Active Transactions DMVSELECT * FROM sys.dm_tran_active_transactions;Lists all open active transactions.
141Check Session Transactions DMVSELECT * FROM sys.dm_tran_session_transactions;Maps active transactions to SPIDs.
142Check Lock Escalation ModeSELECT name, lock_escalation_desc FROM sys.tables;Inspects lock escalation settings (TABLE, AUTO, DISABLE).
143Set Lock Escalation DisableALTER TABLE Users SET (LOCK_ESCALATION = DISABLE);Prevents row locks from escalating to table lock.
144Set Lock Escalation AutoALTER TABLE Users SET (LOCK_ESCALATION = AUTO);Allows partition-level lock escalation.
145Check Index Usage Stats DMVSELECT OBJECT_NAME(object_id), user_seeks, user_scans, user_lookups, user_updates FROM sys.dm_db_index_usage_stats WHERE database_id = DB_ID();Pulls read vs write I/O per index.
146Check Operational Index Stats DMVSELECT OBJECT_NAME(object_id), page_latch_wait_count, page_io_latch_wait_count FROM sys.dm_db_index_operational_stats(DB_ID(), NULL, NULL, NULL);Monitors latch contention per index page.
147Check Physical Index Stats DMVSELECT OBJECT_NAME(object_id), avg_fragmentation_in_percent, page_count FROM sys.dm_db_index_physical_stats(DB_ID(), NULL, NULL, NULL, 'SAMPLED');Evaluates index fragmentation percentage.
148Check Index Operational Leaf InsertsSELECT OBJECT_NAME(object_id), leaf_insert_count FROM sys.dm_db_index_operational_stats(DB_ID(), NULL, NULL, NULL);Measures leaf page write activity.
149Check Missing Index Groups DMVSELECT * FROM sys.dm_db_missing_index_groups;Maps missing index details to group IDs.
150Check Missing Index Group Stats DMVSELECT * FROM sys.dm_db_missing_index_group_stats ORDER BY avg_user_impact DESC;Ranks missing index recommendations by performance impact.
151Check Query Execution Stats Top CPUSELECT TOP 10 total_worker_time/execution_count AS avg_cpu_time, text FROM sys.dm_exec_query_stats CROSS APPLY sys.dm_exec_sql_text(sql_handle) ORDER BY avg_cpu_time DESC;Finds queries with highest average CPU consumption.
152Check Query Execution Stats Top Logical ReadsSELECT TOP 10 total_logical_reads/execution_count AS avg_logical_reads, text FROM sys.dm_exec_query_stats CROSS APPLY sys.dm_exec_sql_text(sql_handle) ORDER BY avg_logical_reads DESC;Finds queries with highest memory buffer reads.
153Check Query Execution Stats Top Physical ReadsSELECT TOP 10 total_physical_reads/execution_count AS avg_physical_reads, text FROM sys.dm_exec_query_stats CROSS APPLY sys.dm_exec_sql_text(sql_handle) ORDER BY avg_physical_reads DESC;Finds queries with highest physical disk I/O.
154Check Cached Execution Plan XMLSELECT TOP 5 query_plan, text FROM sys.dm_exec_cached_plans CROSS APPLY sys.dm_exec_sql_text(plan_handle) CROSS APPLY sys.dm_exec_query_plan(plan_handle);Pulls raw graphical XML execution plans from memory.
155Check Buffer Cache Size per DBSELECT DB_NAME(database_id), COUNT(*)*8/1024 AS BufferSizeMB FROM sys.dm_os_buffer_descriptors GROUP BY database_id;Calculates RAM buffer pool usage per database.
156Check Memory Clerks Top AllocationSELECT TOP 10 type, SUM(pages_kb)/1024 AS SizeMB FROM sys.dm_os_memory_clerks GROUP BY type ORDER BY SUM(pages_kb) DESC;Analyzes memory allocation across internal components.
157Check OS Sys InfoSELECT cpu_count, hyperthread_ratio, physical_memory_kb/1024 AS PhysicalMemoryMB FROM sys.dm_os_sys_info;Inspects CPU cores, threads, and host RAM.
158Check OS Ring Buffers Out Of MemorySELECT * FROM sys.dm_os_ring_buffers WHERE ring_buffer_type = 'RING_BUFFER_RESOURCE_MONITOR';Monitors system and process low-memory events.
159Check Server PropertiesSELECT SERVERPROPERTY('MachineName'), SERVERPROPERTY('InstanceName'), SERVERPROPERTY('Edition'), SERVERPROPERTY('ProductVersion');Pulls host, instance, edition, and build version.
160Check Current Session ContextSELECT @@SPID AS SessionID, SYSTEM_USER AS SystemUser, DB_NAME() AS ActiveDatabase;Pulls current session SPID, user, and database name.
161Check Server UptimeSELECT sqlserver_start_time FROM sys.dm_os_sys_info;Outputs exact SQL Server boot timestamp.
162Check SQL Agent Service StatusSELECT status_desc FROM sys.dm_server_services WHERE servicename LIKE 'SQL Server Agent%';Verifies if SQL Agent service is running.
163Check Database Master Key TDESELECT * FROM sys.symmetric_keys WHERE name LIKE '%DatabaseMasterKey%';Verifies master encryption key for TDE.
164Check AlwaysOn Replica HealthSELECT replica_server_name, operational_state_desc, connected_state_desc, synchronization_health_desc FROM sys.dm_hadr_availability_replica_states;Inspects AG replica cluster health.
165Check AlwaysOn Database StateSELECT database_name, synchronization_state_desc, synchronization_health_desc FROM sys.dm_hadr_database_replica_cluster_states;Inspects AG database synchronization state.
166Check Service Broker QueuesSELECT name, is_enqueue_enabled, is_dequeue_enabled FROM sys.service_queues;Inspects Service Broker message queues.
167Check Service Broker EndpointsSELECT name, state_desc, port FROM sys.endpoints WHERE type = 3;Inspects Service Broker network endpoint port.
168Check Database Mirroring EndpointsSELECT name, state_desc, port FROM sys.endpoints WHERE type = 4;Inspects Database Mirroring network endpoint port.
169Check Database Mirroring StatusSELECT db_name(database_id), mirroring_state_desc, mirroring_role_desc FROM sys.database_mirroring WHERE mirroring_guid IS NOT NULL;Inspects legacy database mirroring status.
170Check Change Tracking Table StatusSELECT OBJECT_NAME(object_id), is_track_columns_updated_on FROM sys.change_tracking_tables;Inspects Change Tracking enabled tables.
171Check CDC Capture JobsSELECT * FROM msdb.dbo.cdc_jobs;Lists active CDC capture and cleanup jobs.
172Check Extended Events Buffer TargetSELECT * FROM sys.dm_xe_session_targets;Inspects event ring buffer targets for XE.
173Check Extended Events PackagesSELECT name, description FROM sys.xe_packages;Lists installed Extended Events packages.
174Check Extended Events Event NamesSELECT name, description FROM sys.xe_objects WHERE object_type = 'event' AND name LIKE 'sql%';Lists available SQL Server Extended Events.
175Check User Defined Functions ListSELECT name, type_desc FROM sys.objects WHERE type IN ('FN', 'IF', 'TF');Lists user defined functions in database.
176Check Stored Procedures ListSELECT name, create_date, modify_date FROM sys.procedures;Lists stored procedures in database.
177Check Views ListSELECT name, create_date FROM sys.views;Lists views in database.
178Check Triggers ListSELECT name, OBJECT_NAME(parent_id) AS TableName, is_disabled FROM sys.triggers;Lists triggers in database.
179Check Fulltext CatalogsSELECT name, is_default FROM sys.fulltext_catalogs;Lists full-text search catalogs.
180Check Fulltext IndexesSELECT OBJECT_NAME(object_id), is_enabled FROM sys.fulltext_indexes;Lists full-text search indexes.
181Check Database Filestream OptionSELECT name, non_transacted_access_desc FROM sys.databases WHERE name = 'EnterpriseDB';Inspects FILESTREAM feature settings.
182Check Filestream FilegroupsSELECT name, type_desc FROM sys.filegroups WHERE type = 'FD';Lists FILESTREAM storage filegroups.
183Check Columnstore Rowgroups CompressionSELECT OBJECT_NAME(object_id), row_group_id, state_desc, total_rows FROM sys.dm_db_column_store_row_group_physical_stats;Inspects Columnstore rowgroup compression states.
184Check In-Memory Container MemorySELECT memory_consumer_type_desc, alloc_size_mb FROM sys.dm_db_xtp_memory_consumers;Analyzes memory allocation for Hekaton in-memory tables.
185Check In-Memory TransactionsSELECT * FROM sys.dm_db_xtp_transactions;Monitors active transactions on in-memory tables.
186Check Database Read Only SettingSELECT name, is_read_only FROM sys.databases WHERE name = 'EnterpriseDB';Inspects read-only setting.
187Set Database Read OnlyALTER DATABASE EnterpriseDB SET READ_ONLY WITH ROLLBACK IMMEDIATE;Forces database into read-only mode.
188Set Database Read WriteALTER DATABASE EnterpriseDB SET READ_WRITE;Restores database to read-write mode.
189Check Database Single User SettingSELECT name, user_access_desc FROM sys.databases WHERE name = 'EnterpriseDB';Inspects user access mode (MULTI_USER, SINGLE_USER, RESTRICTED_USER).
190Set Database Single UserALTER DATABASE EnterpriseDB SET SINGLE_USER WITH ROLLBACK IMMEDIATE;Forces database into single user mode for maintenance.
191Set Database Multi UserALTER DATABASE EnterpriseDB SET MULTI_USER;Restores database to multi-user mode.
192Check Database Parameterization SettingSELECT name, is_parameterization_forced FROM sys.databases WHERE name = 'EnterpriseDB';Inspects forced parameterization setting.
193Set Forced ParameterizationALTER DATABASE EnterpriseDB SET PARAMETERIZATION FORCED;Enforces automatic query parameterization.
194Set Simple ParameterizationALTER DATABASE EnterpriseDB SET PARAMETERIZATION SIMPLE;Restores default simple query parameterization.
195Check Database ANSI NULLS SettingSELECT name, is_ansi_nulls_on FROM sys.databases WHERE name = 'EnterpriseDB';Inspects ANSI NULLS default setting.
196Check Database QUOTED IDENTIFIER SettingSELECT name, is_quoted_identifier_on FROM sys.databases WHERE name = 'EnterpriseDB';Inspects QUOTED_IDENTIFIER default setting.
197Check Database Cursor Default SettingSELECT name, is_cursor_close_on_commit_on FROM sys.databases WHERE name = 'EnterpriseDB';Inspects cursor close on commit setting.
198Check Database Page Verify MethodSELECT name, page_verify_option_desc FROM sys.databases WHERE name = 'EnterpriseDB';Inspects page verify option (CHECKSUM, TORN_PAGE_DETECTION, NONE).
199Verify Product VersionSELECT @@VERSION;Outputs full SQL Server version, architecture, and build string.
200Check Ping QuerySELECT 1;Simple connection ping query.

Oracle Database OLTP

Technical Architecture & Overview

Oracle Database is a enterprise multi-model RDBMS featuring high-performance Multitenant architecture (Container DB / Pluggable DB), Real Application Clusters (RAC) scale-out shared-everything clustering, Automatic Storage Management (ASM), Data Guard replication, and RMAN recovery manager.

Primary Use Cases: Mission-critical core banking, ERP, global supply chain transactional platforms, and high-availability enterprise applications.

Core Components: System Global Area (SGA), Program Global Area (PGA), Database Writer (DBWn), Log Writer (LGWR), Process Monitor (PMON), System Monitor (SMON), and Multitenant PDBs.

Exhaustive 200-command administrative, diagnostic, and operational matrix for Oracle Database.

#Operation / ActionSQL*Plus / RMAN SyntaxDescription
1Startup InstanceSTARTUP NOMOUNT; ALTER DATABASE MOUNT; ALTER DATABASE OPEN;Boots Oracle database instance.
2Shutdown InstanceSHUTDOWN IMMEDIATE;Halts database instance safely.
3Connect via SQL*Plussqlplus sys/password@ORCL as sysdbaOpens administrative SQL session.
4RMAN Backup DatabaseRMAN> BACKUP DATABASE PLUS ARCHIVELOG;Executes RMAN enterprise backup.
5RMAN Restore DatabaseRMAN> RESTORE DATABASE; RECOVER DATABASE;Restores and recovers files.
6Check Tablespace UsageSELECT tablespace_name, sum(bytes)/1024/1024 FROM dba_free_space GROUP BY tablespace_name;Evaluates storage capacity.
7Add DatafileALTER TABLESPACE users ADD DATAFILE '/u01/app/oracle/oradata/users02.dbf' SIZE 2G;Expands tablespace storage.
8Create UserCREATE USER app_user IDENTIFIED BY SecurePass123;Provisions database user.
9Grant DBA RolesGRANT CONNECT, RESOURCE, DBA TO app_user;Assigns administrative privileges.
10View Active SessionsSELECT sid, serial#, username, status FROM v$session;Inspects active database users.
11Kill SessionALTER SYSTEM KILL SESSION '145,2341';Terminates hung user session.
12Gather Table StatisticsEXEC DBMS_STATS.GATHER_TABLE_STATS(ownname => 'HR', tabname => 'EMPLOYEES');Updates optimizer statistics.
13Explain Execution PlanEXPLAIN PLAN FOR SELECT * FROM employees;Generates query execution plan.
14Check Alert LogSELECT message_text FROM v$diag_alert_ext ORDER BY recorded_time DESC;Inspects system diagnostic alerts.
15Flush Shared PoolALTER SYSTEM FLUSH SHARED_POOL;Clears compiled cursor memory.
16Resize SGA BufferALTER SYSTEM SET sga_target = 8G SCOPE=BOTH;Adjusts memory allocation pool.
17Create IndexCREATE INDEX idx_emp_email ON employees(email) TABLESPACE users;Provisions table index.
18Drop IndexDROP INDEX idx_emp_email;Removes table index.
19Check Redo Log StatusSELECT group#, status, bytes FROM v$log;Inspects transactional redo logs.
20Switch Redo LogALTER SYSTEM SWITCH LOGFILE;Forces active log file rotation.
21Create Table PartitionCREATE TABLE sales (sale_date DATE) PARTITION BY RANGE (sale_date) (...);Provisions partitioned table.
22Check Data Guard SyncSELECT dest_id, status, gap_status FROM v$archive_dest_status;Verifies standby replication sync.
23Create Profile PasswordCREATE PROFILE secure_user LIMIT PASSWORD_LIFE_TIME 90;Enforces password governance policy.
24View Lock WaitersSELECT * FROM v$lock WHERE block > 0;Identifies lock contention sources.
25Enable TracingEXEC DBMS_SESSION.SET_SQL_TRACE_IN_SESSION(sid=>145, serial#=>2341, sql_trace=>TRUE);Activates detailed event tracing.
26Purge Recycle BinPURGE DBA_RECYCLEBIN;Empties dropped table storage.
27Verify Patch LevelSELECT * FROM dba_registry_sqlpatch;Inspects database patch history.
28Create Database LinkCREATE DATABASE LINK remote_db CONNECT TO user IDENTIFIED BY pass USING 'ORCL_REMOTE';Provisions federated database link.
29Check Invalid ObjectsSELECT owner, object_name FROM dba_objects WHERE status = 'INVALID';Identifies broken compiled routines.
30Check Service Statuslsnrctl statusVerifies Oracle TNS listener state.
31Recompile Invalid ObjectsEXEC DBMS_UTILITY.COMPILE_SCHEMA('HR');Recompiles all invalid PL/SQL objects in schema.
32Set PGA TargetALTER SYSTEM SET pga_aggregate_target = 4G SCOPE=BOTH;Adjusts Program Global Area memory target.
33Check Memory AdvisorySELECT * FROM v$sga_target_advice;Provides sizing recommendations for SGA.
34Check PGA AdvisorySELECT * FROM v$pga_target_advice;Provides sizing recommendations for PGA.
35Create Temporary TablespaceCREATE TEMPORARY TABLESPACE temp02 TEMPFILE '/u01/app/oracle/oradata/temp02.dbf' SIZE 1G;Adds secondary temp tablespace.
36Resize DatafileALTER DATABASE DATAFILE '/u01/app/oracle/oradata/users01.dbf' RESIZE 10G;Resizes datafile volume size.
37Enable Autoextend DatafileALTER DATABASE DATAFILE '/u01/app/oracle/oradata/users01.dbf' AUTOEXTEND ON NEXT 100M MAXSIZE 32G;Sets auto-growth settings.
38Drop TablespaceDROP TABLESPACE users INCLUDING CONTENTS AND DATAFILES;Deletes tablespace and files from disk.
39Create Undo TablespaceCREATE UNDO TABLESPACE undotbs2 DATAFILE '/u01/app/oracle/oradata/undotbs02.dbf' SIZE 2G;Provisions new Undo tablespace.
40Switch Undo TablespaceALTER SYSTEM SET undo_tablespace = undotbs2 SCOPE=BOTH;Activates new Undo tablespace.
41Check Undo RetentionSHOW PARAMETER undo_retention;Inspects undo retention duration setting.
42Set Undo RetentionALTER SYSTEM SET undo_retention = 3600 SCOPE=BOTH;Enforces 1-hour undo retention for flashback.
43Check ASM Disk GroupSELECT name, state, total_mb, free_mb FROM v$asm_diskgroup;Inspects ASM storage pool capacity.
44Add ASM DiskALTER DISKGROUP data ADD DISK '/dev/oracleasm/disks/DISK05';Expands ASM disk group storage.
45Drop ASM DiskALTER DISKGROUP data DROP DISK DISK05;Removes disk from ASM storage group.
46Rebalance ASM DiskgroupALTER DISKGROUP data REBALANCE POWER 8;Triggers high-priority ASM rebalance.
47RMAN Crosscheck BackupRMAN> CROSSCHECK BACKUP;Validates backup file presence on disk/tape.
48RMAN Delete ObsoleteRMAN> DELETE OBSOLETE;Purges expired backups per retention policy.
49RMAN Validate DatabaseRMAN> VALIDATE DATABASE;Checks datafiles for physical block corruption.
50Check Block Corruption DMVSELECT * FROM v$database_block_corruption;Lists corrupted data blocks found by RMAN.
51Recover Corrupted BlockRMAN> RECOVER DATAFILE 4 BLOCK 128;Executes online block media recovery.
52Create Pluggable DB (PDB)CREATE PLUGGABLE DATABASE pdb2 ADMIN USER pdb_admin IDENTIFIED BY pass FILE_NAME_CONVERT=('/pdbseed/','/pdb2/');Provisions new Multitenant PDB.
53Plug PDB into CDBCREATE PLUGGABLE DATABASE pdb3 USING '/u01/app/oracle/oradata/pdb3.xml' NOCOPY;Plugs PDB into Container DB.
54Unplug PDB from CDBALTER PLUGGABLE DATABASE pdb2 UNPLUG INTO '/u01/app/oracle/oradata/pdb2.xml';Unplugs PDB into XML metadata file.
55Open PDBALTER PLUGGABLE DATABASE pdb2 OPEN;Boots PDB into read-write state.
56Close PDBALTER PLUGGABLE DATABASE pdb2 CLOSE IMMEDIATE;Shuts down pluggable database.
57Save PDB StateALTER PLUGGABLE DATABASE ALL SAVE STATE;Preserves open state across CDB restarts.
58Drop PDBDROP PLUGGABLE DATABASE pdb2 INCLUDING DATAFILES;Deletes multitenant PDB.
59Check Container DB ListSELECT name, open_mode FROM v$pdbs;Lists all PDBs in CDB.
60Switch Container SessionALTER SESSION SET CONTAINER = pdb1;Changes current session focus to PDB.
61Create Common UserCREATE USER c##app_admin IDENTIFIED BY SecurePass123 CONTAINER=ALL;Provisions CDB-wide common account.
62Grant Common PrivilegeGRANT CREATE SESSION TO c##app_admin CONTAINER=ALL;Grants privilege across all PDBs.
63Check AWR Snapshot ConfigSELECT snap_interval, retention FROM dba_hist_wr_control;Inspects AWR repository collection schedule.
64Modify AWR Snapshot IntervalEXEC DBMS_WORKLOAD_REPOSITORY.MODIFY_SNAPSHOT_SETTINGS(retention=>40320, interval=>30);Sets 28-day retention and 30m snaps.
65Create AWR SnapshotEXEC DBMS_WORKLOAD_REPOSITORY.CREATE_SNAPSHOT();Triggers manual AWR snapshot.
66Generate AWR ReportSELECT * FROM TABLE(DBMS_WORKLOAD_REPOSITORY.AWR_REPORT_HTML(1, 1, 101, 102));Generates HTML performance report.
67Generate ASH ReportSELECT * FROM TABLE(DBMS_WORKLOAD_REPOSITORY.ASH_REPORT_HTML(1, 1, SYSDATE-1/24, SYSDATE));Generates Active Session History report.
68Create SQL Tuning SetEXEC DBMS_SQLTUNE.CREATE_SQLSET('my_sts');Provisions container for SQL tuning.
69Run SQL Tuning AdvisorDECLARE l_task VARCHAR2(30); BEGIN l_task := DBMS_SQLTUNE.CREATE_TUNING_TASK(sql_id=>'43abc123', task_name=>'tune_q1'); DBMS_SQLTUNE.EXECUTE_TUNING_TASK('tune_q1'); END;Runs AI query tuning analysis.
70Get Tuning ReportSELECT DBMS_SQLTUNE.REPORT_TUNING_TASK('tune_q1') FROM DUAL;Fetches recommendations and SQL profiles.
71Accept SQL ProfileEXEC DBMS_SQLTUNE.ACCEPT_SQL_PROFILE(task_name=>'tune_q1', name=>'prof_q1');Applies optimizer plan profile.
72Check Active SQL ExecutionSELECT sql_id, elapsed_time/1e6, cpu_time/1e6 FROM v$sql WHERE executions > 0;Monitors CPU and elapsed time.
73Gather Schema StatisticsEXEC DBMS_STATS.GATHER_SCHEMA_STATS(ownname=>'HR', options=>'GATHER AUTO');Gathers stats for outdated objects.
74Lock Table StatisticsEXEC DBMS_STATS.LOCK_TABLE_STATS(ownname=>'HR', tabname=>'EMPLOYEES');Prevents automatic stats override.
75Unlock Table StatisticsEXEC DBMS_STATS.UNLOCK_TABLE_STATS(ownname=>'HR', tabname=>'EMPLOYEES');Allows automatic stats updates.
76Flashback TableFLASHBACK TABLE employees TO TIMESTAMP (SYSTIMESTAMP - INTERVAL '15' MINUTE);Reverts table state 15 minutes back.
77Flashback DatabaseFLASHBACK DATABASE TO RESTORE POINT before_upgrade;Reverts whole database to restore point.
78Create Restore PointCREATE RESTORE POINT before_upgrade GUARANTEE FLASHBACK DATABASE;Provisions guaranteed restore point.
79Drop Restore PointDROP RESTORE POINT before_upgrade;Deletes restore point.
80Check Flashback LogsSELECT * FROM v$flashback_database_log;Inspects flashback log space utilization.
81Enable Archive Log ModeALTER DATABASE ARCHIVELOG;Enables continuous WAL archiving.
82Disable Archive Log ModeALTER DATABASE NOARCHIVELOG;Disables continuous WAL archiving.
83Check Archive DestinationsSELECT destination, status FROM v$archive_dest;Inspects archive log destinations.
84Check Lock Waits DMVSELECT c.owner, c.object_name, c.object_type, b.sid, b.serial# FROM v$locked_object a, v$session b, dba_objects c WHERE a.session_id = b.sid AND a.object_id = c.object_id;Finds sessions holding locks on objects.
85Check Latches DMVSELECT name, gets, misses, sleeps FROM v$latch ORDER BY sleeps DESC;Inspects latch contention.
86Check Library Cache PinSELECT * FROM v$session_wait WHERE event LIKE 'library cache%';Identifies PL/SQL package lock waits.
87Enable Audit VaultAUDIT ALL BY app_user BY ACCESS;Enables fine-grained audit on account.
88Disable AuditNOAUDIT ALL BY app_user;Stops audit tracking on account.
89View Audit TrailSELECT username, action_name, timestamp FROM dba_audit_trail;Inspects historical audit log entries.
90Check Unified Audit TrailSELECT event_timestamp, dbusername, action_name FROM unified_audit_trail;Inspects 12c+ unified audit log.
91Set Resource Manager PlanALTER SYSTEM SET resource_manager_plan = 'DEFAULT_PLAN' SCOPE=BOTH;Activates CPU resource governor.
92Create Resource Consumer GroupEXEC DBMS_RESOURCE_MANAGER.CREATE_CONSUMER_GROUP(consumer_group=>'batch_grp', comment=>'Batch tasks');Provisions CPU consumer group.
93Grant Consumer GroupEXEC DBMS_RESOURCE_MANAGER_PRIVS.GRANT_SWITCH_CONSUMER_GROUP('app_user', 'batch_grp', FALSE);Assigns user to consumer group.
94Switch Consumer GroupEXEC DBMS_SESSION.SWITCH_CURRENT_CONSUMER_GROUP('batch_grp', l_old_group, FALSE);Changes session CPU priority.
95Export Schema via Data Pumpexpdp hr/pass DIRECTORY=dp_dir DUMPFILE=hr.dmp SCHEMAS=HR;Exports schema binary via Data Pump.
96Import Schema via Data Pumpimpdp hr/pass DIRECTORY=dp_dir DUMPFILE=hr.dmp SCHEMAS=HR;Imports schema binary via Data Pump.
97Kill Data Pump Jobimpdp hr/pass ATTACH=SYS_IMPORT_SCHEMA_01 -> KILL_JOBStops running Data Pump operation.
98Verify Database VersionSELECT * FROM v$version;Outputs Oracle DB release and patch level.
99Check Oracle License UsageSELECT * FROM v$license;Inspects active session count vs high-water mark.
100Check Flash Recovery AreaSELECT * FROM v$recovery_file_dest;Inspects FRA disk space and usage percentage.
101Check RAC Cluster InterconnectSELECT * FROM v$cluster_interconnects;Inspects private network interfaces for RAC nodes.
102Check RAC Node StatusSELECT inst_id, instance_name, host_name, status FROM gv$instance;Inspects status of all RAC cluster instances.
103Check RAC Lock Ping LatencySELECT * FROM v$gc_element WHERE flags > 0;Monitors Global Cache lock element transfers.
104Check ASM Disk Rebalance StatusSELECT * FROM v$asm_operation;Monitors background ASM disk rebalance progress.
105Check Flashback Database StatusSELECT flashback_on FROM v$database;Verifies if Flashback Database logging is enabled.
106Check Flashback Area UsageSELECT name, space_limit/1024/1024/1024 AS limit_gb, space_used/1024/1024/1024 AS used_gb FROM v$recovery_file_dest;Monitors FRA storage.
107Check Data Guard LagSELECT name, value FROM v$dataguard_stats WHERE name LIKE '%lag%';Measures transport and apply lag on standby DB.
108Set Data Guard Protection ModeALTER DATABASE SET STANDBY DATABASE TO MAXIMIZE PROTECTION;Enforces zero-data-loss synchronous Data Guard.
109Check Redo Log Thread GroupSELECT thread#, group#, bytes, status FROM v$log;Inspects redo log threads across RAC instances.
110Add Redo Log File GroupALTER DATABASE ADD LOGFILE GROUP 4 ('/u01/app/oracle/oradata/redo04.log') SIZE 1G;Adds 1GB redo log file group.
111Drop Redo Log File GroupALTER DATABASE DROP LOGFILE GROUP 4;Removes empty redo log group.
112Check Table Partition BoundSELECT partition_name, high_value FROM dba_tab_partitions WHERE table_name = 'SALES';Inspects boundary values of table partitions.
113Truncate Table PartitionALTER TABLE sales TRUNCATE PARTITION p2026_q1;Clears partition records instantly.
114Merge Table PartitionsALTER TABLE sales MERGE PARTITIONS p2026_q1, p2026_q2 INTO PARTITION p2026_h1;Merges adjacent partitions.
115Split Table PartitionALTER TABLE sales SPLIT PARTITION p2026_h1 AT (TO_DATE('2026-04-01','YYYY-MM-DD')) INTO (PARTITION p2026_q1, PARTITION p2026_q2);Splits partition.
116Check Index Fragmentation OracleSELECT index_name, blevel, leaf_blocks, distinct_keys FROM dba_indexes WHERE table_name = 'EMPLOYEES';Evaluates B-Tree index depth and blocks.
117Coalesce IndexALTER INDEX idx_emp_email COALESCE;Defragments index leaf blocks in-place.
118Rebuild Index OnlineALTER INDEX idx_emp_email REBUILD ONLINE;Rebuilds index without locking concurrent DML writes.
119Check Table Degree of ParallelismSELECT table_name, degree FROM dba_tables WHERE table_name = 'EMPLOYEES';Inspects table parallel execution degree.
120Set Table Parallel ExecutionALTER TABLE employees PARALLEL 8;Enforces 8 parallel execution threads for queries.
121Check SQL Execution Cursor CacheSELECT sql_text, executions, disk_reads, buffer_gets FROM v$sqlarea WHERE executions > 100 ORDER BY disk_reads DESC;Finds queries with high physical I/O.
122Check Active Session History (ASH)SELECT sample_time, session_id, sql_id, event FROM v$active_session_history WHERE sample_time > SYSDATE-1/24;Analyzes wait events in last hour.
123Check System Wait EventsSELECT event, total_waits, time_waited FROM v$system_event ORDER BY time_waited DESC;Pulls top system-wide wait events.
124Check Enqueue Lock WaitsSELECT * FROM v$enqueue_stat WHERE failed > 0;Monitors internal lock enqueue failures.
125Check Library Cache MemorySELECT * FROM v$librarycache;Monitors cursor parsing and execution hit ratios.
126Check Temp Tablespace Space UsageSELECT tablespace_name, allocated_space, used_space FROM v$temp_space_header;Inspects temp tablespace page allocations.
127Check Active Rollback Undo SegmentsSELECT segment_name, status, bytes FROM dba_undo_extents;Inspects Undo segment allocations.
128Check Oracle Database NameSELECT name, db_unique_name, open_mode FROM v$database;Outputs database name and open state.
129Check Oracle Instance NameSELECT instance_name, host_name, version FROM v$instance;Outputs instance name and host info.
130Verify Listener Status CLIlsnrctl statusVerifies TNS network listener process status.
131Check Oracle Database RoleSELECT database_role FROM v$database;Inspects PRIMARY vs STANDBY role.
132Check Oracle Protection ModeSELECT protection_mode, protection_level FROM v$database;Inspects Data Guard protection level.
133Check Oracle Log ModeSELECT log_mode FROM v$database;Inspects ARCHIVELOG vs NOARCHIVELOG mode.
134Check Oracle Switchover StatusSELECT switchover_status FROM v$database;Inspects Data Guard switchover readiness.
135Check Oracle SGA Dynamic ComponentsSELECT component, current_size/1024/1024 AS size_mb FROM v$sga_dynamic_components;Monitors SGA component sizing.
136Check Oracle Buffer Cache Hit RatioSELECT (1 - (phy.value / (cur.value + con.value))) * 100 AS Hit_Ratio FROM v$sysstat phy, v$sysstat cur, v$sysstat con WHERE phy.name = 'physical reads' AND cur.name = 'db block gets' AND con.name = 'consistent gets';Calculates buffer cache hit ratio.
137Check Oracle Shared Pool Free MemorySELECT name, bytes/1024/1024 AS free_mb FROM v$sgastat WHERE name = 'free memory';Inspects unallocated shared pool memory.
138Check Oracle Library Cache Hit RatioSELECT sum(pins)/(sum(pins)+sum(reloads))*100 AS Hit_Ratio FROM v$librarycache;Calculates library cache cursor execution hit ratio.
139Check Oracle Dictionary Cache Hit RatioSELECT (1 - (sum(getmisses)/sum(gets)))*100 AS Hit_Ratio FROM v$rowcache;Calculates data dictionary cache hit ratio.
140Check Oracle Undo Retention GuaranteeSELECT tablespace_name, retention FROM dba_tablespaces WHERE contents = 'UNDO';Inspects UNDO retention guarantee setting.
141Check Oracle Undo Space UsageSELECT sum(bytes)/1024/1024 AS total_undo_mb FROM dba_data_files WHERE tablespace_name LIKE 'UNDO%';Calculates allocated Undo tablespace capacity.
142Check Oracle Temp Space UsageSELECT tablespace_name, sum(bytes_cached)/1024/1024 AS used_temp_mb FROM v$temp_extent_pool GROUP BY tablespace_name;Calculates temp space usage.
143Check Oracle ASM Disks StateSELECT mount_status, header_status, mode_status, path FROM v$asm_disk;Inspects ASM disk headers and mount state.
144Check Oracle ASM Diskgroup BalanceSELECT group_number, name, state, type, total_mb, free_mb FROM v$asm_diskgroup;Inspects ASM diskgroup total vs free capacity.
145Check Oracle RMAN Backup StatusSELECT operation, status, start_time, end_time FROM v$rman_status WHERE start_time > SYSDATE-7 ORDER BY start_time DESC;Inspects RMAN job statuses for last 7 days.
146Check Oracle RMAN Backup ProgressSELECT sid, serial#, opname, sofar, totalwork, round(sofar/totalwork*100,2) AS pct FROM v$session_longops WHERE opname LIKE 'RMAN%' AND totalwork > 0;Monitors active RMAN backup progress.
147Check Oracle Datafile Header StatusSELECT file#, name, status, error FROM v$datafile_header;Inspects datafile header consistency.
148Check Oracle Control Files ListSELECT name FROM v$controlfile;Lists active control file paths.
149Check Oracle Redo Log Member FilesSELECT group#, member, status FROM v$logfile;Lists redo log member paths and mirror status.
150Check Oracle Archived Logs ListSELECT thread#, sequence#, first_change#, next_change#, completion_time FROM v$archived_log WHERE completion_time > SYSDATE-1 ORDER BY sequence# DESC;Lists archived log files generated in last 24h.
151Check Oracle Pluggable DB Open ModeSELECT name, open_mode, restricted FROM v$pdbs;Lists PDBs and open status.
152Check Oracle PDB Save StateSELECT instance_name, pdb_name, state FROM cdb_pdb_saved_states;Inspects PDB preserved startup state.
153Check Oracle PDB Resource ManagerSELECT plan, status FROM dba_cdb_rsrc_plans;Inspects multitenant resource manager plan.
154Check Oracle Tablespace EncryptionSELECT tablespace_name, encrypted FROM dba_tablespaces;Inspects TDE encrypted tablespaces.
155Check Oracle Wallet StatusSELECT status, wallet_type FROM v$encryption_wallet;Inspects TDE master key wallet status.
156Check Oracle Tablespace AutoextendSELECT tablespace_name, file_name, autoextensible, maxbytes/1024/1024 AS max_mb FROM dba_data_files;Inspects auto-growth limits.
157Check Oracle Segment Size Top 10SELECT * FROM (SELECT owner, segment_name, segment_type, bytes/1024/1024 AS size_mb FROM dba_segments ORDER BY bytes DESC) WHERE ROWNUM <= 10;Finds 10 largest storage segments in database.
158Check Oracle Table Row Count StatsSELECT table_name, num_rows, blocks, last_analyzed FROM dba_tables WHERE owner = 'HR';Inspects table optimizer row count stats.
159Check Oracle Index Clustering FactorSELECT index_name, clustering_factor, leaf_blocks FROM dba_indexes WHERE owner = 'HR';Evaluates index data layout alignment with table rows.
160Check Oracle Stored Code ErrorsSELECT name, type, line, text FROM dba_errors WHERE owner = 'HR';Pulls PL/SQL compilation error messages.
161Check Oracle System Privilege GrantsSELECT grantee, privilege FROM dba_sys_privs WHERE grantee = 'APP_USER';Lists system privileges assigned to user.
162Check Oracle Role GrantsSELECT grantee, granted_role FROM dba_role_privs WHERE grantee = 'APP_USER';Lists roles assigned to user.
163Check Oracle Object Privilege GrantsSELECT grantee, table_name, privilege FROM dba_tab_privs WHERE grantee = 'APP_USER';Lists object privileges assigned to user.
164Check Oracle Password Profile LimitsSELECT profile, resource_name, limit FROM dba_profiles WHERE resource_type = 'PASSWORD';Inspects password policy rules.
165Check Oracle User Profile AssignmentSELECT username, profile, account_status FROM dba_users WHERE username = 'APP_USER';Inspects assigned profile for account.
166Check Oracle Audit PoliciesSELECT policy_name, enabled FROM audit_unified_enabled_policies;Lists active unified audit policies.
167Check Oracle Audit Trail EventsSELECT event_timestamp, dbusername, action_name, return_code FROM unified_audit_trail WHERE event_timestamp > SYSDATE-1 ORDER BY event_timestamp DESC;Pulls recent audit events.
168Check Oracle Scheduler JobsSELECT job_name, state, last_start_date, next_run_date FROM dba_scheduler_jobs;Lists DBMS_SCHEDULER jobs.
169Check Oracle Scheduler Job LogsSELECT job_name, status, log_date FROM dba_scheduler_job_run_details WHERE log_date > SYSDATE-1 ORDER BY log_date DESC;Inspects DBMS_SCHEDULER execution logs.
170Run Oracle Scheduler JobEXEC DBMS_SCHEDULER.RUN_JOB('NIGHTLY_BACKUP');Triggers immediate execution of DBMS_SCHEDULER job.
171Check Oracle Database Parameter non-defaultSELECT name, value FROM v$parameter WHERE isdefault = 'FALSE';Lists non-default initialization parameters.
172Check Oracle SPFile LocationSELECT value FROM v$parameter WHERE name = 'spfile';Outputs path to active SPFILE binary.
173Check Oracle Session CPU UsageSELECT s.sid, s.username, ss.value/100 AS cpu_sec FROM v$session s JOIN v$sesstat ss ON s.sid = ss.sid JOIN v$statname sn ON ss.statistic# = sn.statistic# WHERE sn.name = 'CPU used by this session' ORDER BY ss.value DESC;Finds sessions using highest CPU time.
174Check Oracle Session Memory UsageSELECT s.sid, s.username, ss.value/1024/1024 AS pga_mb FROM v$session s JOIN v$sesstat ss ON s.sid = ss.sid JOIN v$statname sn ON ss.statistic# = sn.statistic# WHERE sn.name = 'session pga memory' ORDER BY ss.value DESC;Finds sessions using highest PGA memory.
175Check Oracle Long Running OperationsSELECT sid, serial#, opname, target_desc, sofar, totalwork, round(sofar/totalwork*100,2) AS pct FROM v$session_longops WHERE totalwork > 0 AND sofar < totalwork;Monitors long-running queries and utilities.
176Check Oracle Query PingSELECT 1 FROM DUAL;Simple connection ping test returning 1.
177Check Oracle Current UserSELECT USER FROM DUAL;Outputs active execution user account.
178Check Oracle Current ContainerSELECT SYS_CONTEXT('USERENV', 'CON_NAME') FROM DUAL;Outputs active container PDB name.
179Check Oracle Current DB NameSELECT SYS_CONTEXT('USERENV', 'DB_NAME') FROM DUAL;Outputs active database name.
180Check Oracle Current HostnameSELECT SYS_CONTEXT('USERENV', 'SERVER_HOST') FROM DUAL;Outputs host server hostname.
181Check Oracle Current TimeSELECT SYSDATE FROM DUAL;Outputs current database server date and time.
182Check Oracle Current TimestampSELECT SYSTIMESTAMP FROM DUAL;Outputs current database server timestamp with timezone.
183Check Oracle Database VersionSELECT * FROM v$version;Outputs full Oracle release and build version string.
184Check Oracle Instance Startup TimeSELECT startup_time FROM v$instance;Outputs exact instance boot timestamp.
185Check Oracle Listener Port TNSSELECT value FROM v$parameter WHERE name = 'local_listener';Inspects local TNS listener parameter.
186Check Oracle Database Character SetSELECT value FROM nls_database_parameters WHERE parameter = 'NLS_CHARACTERSET';Outputs database character set encoding.
187Check Oracle Database NLS TerritorySELECT value FROM nls_database_parameters WHERE parameter = 'NLS_TERRITORY';Outputs database territory NLS setting.
188Check Oracle Database NLS LanguageSELECT value FROM nls_database_parameters WHERE parameter = 'NLS_LANGUAGE';Outputs database language NLS setting.
189Check Oracle Service NamesSELECT name FROM v$active_services;Lists active network service names registered with listener.
190Check Oracle Dispatchers StatusSELECT name, status, accept FROM v$dispatcher;Inspects shared server dispatcher status.
191Check Oracle Instance StatusSELECT instance_name, status FROM v$instance;Inspects instance status.
192Check Oracle Database NameSELECT name FROM v$database;Inspects database name.
193Check Oracle Log ModeSELECT log_mode FROM v$database;Inspects archivelog mode.
194Check Oracle Open ModeSELECT open_mode FROM v$database;Inspects database open mode.
195Check Oracle DBIDSELECT dbid FROM v$database;Inspects unique database ID.
196Check Oracle Platform NameSELECT platform_name FROM v$database;Inspects OS platform name.
197Check Oracle Controlfile TypeSELECT controlfile_type FROM v$database;Inspects controlfile type.
198Check Oracle Guard StatusSELECT guard_status FROM v$database;Inspects Data Guard protection status.
199Check Oracle Flashback StatusSELECT flashback_on FROM v$database;Inspects flashback status.
200Check Oracle Instance StartupSELECT startup_time FROM v$instance;Inspects instance boot time.

IBM Db2 z/OS OLTP

Technical Architecture & Overview

IBM Db2 z/OS is a enterprise database management system engineered for IBM Z mainframes. It features extreme availability, tight integration with z/OS (WLM, Parallel Sysplex, Cross-System Coupling Facility), massive scalability, and hybrid transaction/analytical processing (HTAP) with the IBM Db2 Analytics Accelerator (IDAA).

Primary Use Cases: Core mainframe transaction processing, banking ledger systems, insurance claim processing, and enterprise data warehousing.

Core Components: DSN1 Subsystem, Buffer Pools (BP0-BP80), Group Buffer Pools (GBP), Distributed Data Facility (DDF), VSAM Datasets, and System Catalog SYSIBM.

Exhaustive 200-command administrative, diagnostic, and operational matrix for IBM Db2 z/OS.

#Operation / ActionDb2 Console / SQL SyntaxDescription
1Start Db2 Subsystem-START DB2 SUBSYS(DSN1)Boots Db2 z/OS instance.
2Stop Db2 Subsystem-STOP DB2 ACCESS(FORCE)Halts Db2 z/OS instance.
3Display Subsystem Status-DISPLAY SUBSYSInspects operational subsystem state.
4Display Active Threads-DISPLAY THREAD(*)Views connected application threads.
5Cancel Thread-CANCEL THREAD(1245)Terminates active thread.
6Start Database Space-START DATABASE(DSNDB04) SPACENAM(USERTBL) ACCESS(RW)Opens tablespace read-write.
7Stop Database Space-STOP DATABASE(DSNDB04) SPACENAM(USERTBL)Closes tablespace access.
8Run Image Copy UtilityCOPY TABLESPACE DSNDB04.USERTBL COPYDDN(COPY1)Backs up Db2 tablespace.
9Run Reorg UtilityREORG TABLESPACE DSNDB04.USERTBLDefragments dataset storage.
10Run Runstats UtilityRUNSTATS TABLESPACE DSNDB04.USERTBL TABLE(ALL) INDEX(ALL)Refreshes optimizer statistics.
11Run Check IndexCHECK INDEX (DSNDB04.IX01)Validates index consistency.
12Create DatabaseCREATE DATABASE PRODDB BUFFERPOOL BP1;Provisions Db2 database.
13Create TablespaceCREATE TABLESPACE USERTBL IN PRODDB SEGSIZE 32;Provisions logical tablespace.
14Create TableCREATE TABLE USERTBL.EMPLOYEES (EMPNO CHAR(6) NOT NULL);Provisions table schema.
15Create IndexCREATE UNIQUE INDEX USERTBL.XEMP1 ON USERTBL.EMPLOYEES(EMPNO);Provisions unique index.
16Grant AuthorityGRANT SELECT ON TABLE USERTBL.EMPLOYEES TO PUBLIC;Assigns table access rights.
17Explain PlanEXPLAIN STMT; SELECT * FROM USERTBL.EMPLOYEES;Generates query optimization plan.
18Display Archive Log-DISPLAY LOGInspects active logging status.
19Archive Log Switch-ARCHIVE LOGForces active log file archive.
20Display Traces-DISPLAY TRACE(*)Inspects performance monitor traces.
21Start Trace-START TRACE (PERF) CLASS(1,2,3)Activates performance data collection.
22Stop Trace-STOP TRACE (PERF)Halts performance trace collection.
23Display Buffer Pools-DISPLAY BUFFERPOOL(BP1) DETAILInspects buffer pool cache hit ratios.
24Alter Buffer Pool Size-ALTER BUFFERPOOL(BP1) VPSIZE(50000)Modifies virtual buffer pool RAM.
25Recover Table SpaceRECOVER TABLESPACE PRODDB.USERTBL TOCOPY LASTRecovers corrupted tablespace.
26Bind Application PackageBIND PACKAGE(PRODCOLL) MEMBER(PRODPROG) QUALIFIER(PRODSCH)Binds static SQL application package.
27Free PackageFREE PACKAGE(PRODCOLL.PRODPROG(V1))Removes application package.
28Check Workfile Usage-DISPLAY FUNCTION SPECIFIC(*)Inspects sorting workfile usage.
29Display Storage GroupsSELECT SGNAME FROM SYSIBM.SYSSTOGROUP;Lists VSAM storage groups.
30Display System Parameters-DISPLAY ZPARMInspects subsystem tuning parameters.
31Start Database Read-Only-START DATABASE(PRODDB) SPACENAM(*) ACCESS(RO)Opens database in read-only mode.
32Start Database Utility Mode-START DATABASE(PRODDB) SPACENAM(*) ACCESS(UT)Opens database for utility jobs only.
33Display Database Status-DISPLAY DATABASE(PRODDB) SPACENAM(*)Inspects operational status of tablespaces.
34Display Restricted Spaces-DISPLAY DATABASE(*) SPACENAM(*) RESTRICTLists tablespaces in advisory or execution lock.
35Modify Buffer Pool Sequential Threshold-ALTER BUFFERPOOL(BP1) VPSEQT(80)Adjusts sequential prefetch threshold.
36Display Group Buffer Pool-DISPLAY BUFFERPOOL(GBP1) GBPDB2Inspects data sharing group buffer pool.
37Start Trace Accounting-START TRACE (ACCT) CLASS(1,2,3) DEST(SMF)Activates SMF accounting data logging.
38Stop Trace Accounting-STOP TRACE (ACCT)Halts SMF accounting data collection.
39Display Location-DISPLAY LOCATIONInspects distributed DDF connection locations.
40Start DDF Daemon-START DDFBoots Distributed Data Facility daemon.
41Stop DDF Daemon-STOP DDF MODE(QUIESCE)Gracefully stops DDF networking service.
42Display DDF Detail-DISPLAY DDF DETAILInspects active TCP/IP port and thread usage.
43Cancel DDF Thread-CANCEL DDF THREAD(1245)Terminates remote network SQL thread.
44Run Load UtilityLOAD DATA INFILE 'DSN.DATA' INTO TABLE USERTBL.EMPLOYEESBulk ingests sequential dataset into table.
45Run Unload UtilityUNLOAD TABLESPACE PRODDB.USERTBL PUNCHDDN(SYSPUNCH) UNLDDN(SYSREC)Exports tablespace rows to dataset.
46Run Check Data UtilityCHECK DATA TABLESPACE PRODDB.USERTBL FOR EXCEPTION DELETEValidates referential integrity constraint rules.
47Run Repair UtilityREPAIR SET TABLESPACE PRODDB.USERTBL NOCHECKPENDResets check pending status flags.
48Run Quiesce UtilityQUIESCE TABLESPACE PRODDB.USERTBLForces point-in-time write commit across tablespaces.
49Run Modify UtilityMODIFY RECOVERY TABLESPACE PRODDB.USERTBL DELETE AGE(30)Purges obsolete image copy records from SYSCOPY.
50Display RLST-DISPLAY RLSTInspects Resource Limit Specification Table (governor).
51Start RLST Governor-START RLST ID=01Activates SQL CPU governor limits.
52Stop RLST Governor-STOP RLSTDeactivates SQL CPU governor limits.
53Create Storage GroupCREATE STOGROUP SGPROD VOLUMES ('VOL01','VOL02') VCAT DSN1;Provisions VSAM storage group.
54Alter Storage GroupALTER STOGROUP SGPROD ADD VOLUMES ('VOL03');Appends volume serial to storage group.
55Drop Storage GroupDROP STOGROUP SGPROD;Deletes VSAM storage group definition.
56Create Table Space PartitionedCREATE TABLESPACE TS01 IN PRODDB NUMPARTS 4 (PART 1 BUFFERPOOL BP1);Provisions partitioned tablespace.
57Alter Table Space Max partitionsALTER TABLESPACE PRODDB.TS01 MAXPARTITIONS 64;Converts tablespace to Partition-By-Growth.
58Alter Table CompressALTER TABLESPACE PRODDB.TS01 COMPRESS YES;Enforces hardware dictionary data compression.
59Create Index PartitionedCREATE INDEX USERTBL.XEMP1 ON USERTBL.EMPLOYEES(EMPNO) PARTITION BY PART 1 VALUES ('100000');Provisions partitioned index.
60Alter Index RebuildALTER INDEX USERTBL.XEMP1 REBUILD;Rebuilds index tree structure.
61Grant SYSADM AuthorityGRANT SYSADM TO USER JLAWREN;Assigns full Db2 system admin authority.
62Grant DBADM AuthorityGRANT DBADM ON DATABASE PRODDB TO USER JLAWREN;Assigns database administrator rights.
63Revoke SYSADM AuthorityREVOKE SYSADM FROM USER JLAWREN;Revokes system admin authority.
64Display Utility Job-DISPLAY UTILITY(*)Inspects active running batch utilities.
65Term Utility Job-TERM UTILITY(JLAWREN.JOB100)Terminates hung or failed utility job.
66Restart Utility JobRESTART UTILITY(JLAWREN.JOB100)Resumes interrupted utility job from last checkpoint.
67Bind PlanBIND PLAN(PLANPROD) PKLIST(PRODCOLL.*) ACTION(REPLACE)Binds static packages into executable plan.
68Free PlanFREE PLAN(PLANPROD)Deletes static plan from subsystem.
69Rebind PackageREBIND PACKAGE(PRODCOLL.PRODPROG)Recompiles static SQL package against new stats.
70Rebind PlanREBIND PLAN(PLANPROD)Recompiles static execution plan.
71Check Catalog TablesSELECT NAME, CREATOR FROM SYSIBM.SYSTABLES WHERE DBName='PRODDB';Inspects Db2 system catalog.
72Check Catalog ColumnsSELECT NAME, COLTYPE, LENGTH FROM SYSIBM.SYSCOLUMNS WHERE TBNAME='EMPLOYEES';Inspects column definitions in catalog.
73Check Catalog IndexesSELECT NAME, TBNAME, UNIQUERULE FROM SYSIBM.SYSINDEXES WHERE TBNAME='EMPLOYEES';Inspects index definitions in catalog.
74Check Catalog TablespacesSELECT NAME, DBNAME, NTOTAL FROM SYSIBM.SYSTABLESPACE WHERE DBNAME='PRODDB';Inspects tablespace storage pages.
75Check Catalog CopiesSELECT DSNAME, DATE, START_LBN FROM SYSIBM.SYSCOPY WHERE DBNAME='PRODDB';Inspects image copy backup history.
76Check Catalog PackagesSELECT NAME, COLLID, VERSION FROM SYSIBM.SYSPACKAGE WHERE COLLID='PRODCOLL';Inspects bound static packages.
77Check Catalog PlansSELECT NAME, CREATOR, BOUNDTIME FROM SYSIBM.SYSPLAN WHERE NAME='PLANPROD';Inspects bound static plans.
78Check Catalog Volume SpaceSELECT NAME, SPACE FROM SYSIBM.SYSTABLEPART WHERE DBNAME='PRODDB';Inspects partition disk page usage.
79Check Catalog LocksSELECT * FROM SYSIBM.SYSLOCKS;Inspects active subsystem locks.
80Set Current SQLIDSET CURRENT SQLID = 'PRODGRP';Changes active primary authorization ID for DDL.
81Set Current DegreeSET CURRENT DEGREE = 'ANY';Enables parallel processing for dynamic SQL.
82Set Current RulesSET CURRENT RULES = 'DB2';Enforces native Db2 syntax rules.
83Set Current PrecisionSET CURRENT PRECISION = 'DEC31';Configures BCD decimal math precision.
84Display Archive Log Status-DISPLAY ARCHIVEInspects status of archive log datasets.
85Display Profile-DISPLAY PROFILEInspects active performance profiling configurations.
86Start Profile-START PROFILEActivates dynamic SQL performance monitoring.
87Stop Profile-STOP PROFILEHalts dynamic SQL performance monitoring.
88Display Accelerator-DISPLAY ACCEL(*)Inspects IBM Db2 Analytics Accelerator (IDAA) hardware.
89Start Accelerator-START ACCEL(IDAA01)Boots IDAA analytics accelerator appliance.
90Stop Accelerator-STOP ACCEL(IDAA01)Halts IDAA analytics accelerator appliance.
91Display Function-DISPLAY FUNCTION(*)Inspects active user-defined routines.
92Start Function-START FUNCTION(PRODSCH.*)Enables user-defined function execution.
93Stop Function-STOP FUNCTION(PRODSCH.*)Disables user-defined function execution.
94Display Procedure-DISPLAY PROCEDURE(*)Inspects active stored procedures.
95Start Procedure-START PROCEDURE(PRODSCH.*)Enables stored procedure execution.
96Stop Procedure-STOP PROCEDURE(PRODSCH.*)Disables stored procedure execution.
97Display Claimers-DISPLAY DATABASE(PRODDB) CLAIMERSIdentifies threads claiming locks on space.
98Check Locks Detail-DISPLAY DATABASE(PRODDB) LOCKSInspects detailed page and row lock states.
99Check Subsystem VersionSELECT VERSION FROM SYSIBM.SYSENVIRONMENT;Outputs running Db2 z/OS release version.
100Display Storage Group Detail-DISPLAY GROUPInspects data sharing group subsystem state.
101Display DDF Detail Network-DISPLAY DDF DETAILInspects IP addresses, port numbers, and active DDF threads.
102Start DDF Network Service-START DDFBoots Distributed Data Facility network listener.
103Stop DDF Network Service-STOP DDF MODE(QUIESCE)Gracefully stops DDF network connections.
104Display Archive Log Details-DISPLAY LOGInspects active log dataset names and current LSN byte range.
105Display Buffer Pool Detail Stats-DISPLAY BUFFERPOOL(BP1) LIST(*)Inspects page getpages, read I/O, and write I/O per tablespace.
106Display Group Buffer Pool Detail-DISPLAY BUFFERPOOL(GBP1) GBPCACHE(ALL)Inspects coupling facility GBP hit ratios in Data Sharing.
107Display Active Traces-DISPLAY TRACELists active SMF performance and accounting traces.
108Start Performance Trace Class 3-START TRACE(PERF) CLASS(3) DEST(SMF)Activates class 3 performance lock tracing.
109Stop Performance Trace-STOP TRACE(PERF)Halts active performance trace collection.
110Display Database Lock Detail-DISPLAY DATABASE(PRODDB) SPACENAM(*) LOCKSLists page, row, and table locks held across spaces.
111Display Database Claimers-DISPLAY DATABASE(PRODDB) SPACENAM(*) CLAIMERSIdentifies threads holding claims on tablespaces.
112Display Thread Detail-DISPLAY THREAD(*) DETAILOutputs execution details and active SQL text for thread.
113Cancel Thread Force-CANCEL THREAD(1245) FORCEHard-terminates blocking thread immediately.
114Run Image Copy Full UtilityCOPY TABLESPACE PRODDB.TS01 FULL YES COPYDDN(COPY1)Executes full image copy backup to dataset.
115Run Image Copy Incremental UtilityCOPY TABLESPACE PRODDB.TS01 FULL NO COPYDDN(COPY1)Executes incremental image copy backup.
116Run Reorg Table Space UtilityREORG TABLESPACE PRODDB.TS01 SHRLEVEL CHANGEDefragments tablespace without blocking concurrent writes.
117Run Runstats Table Space UtilityRUNSTATS TABLESPACE PRODDB.TS01 TABLE(ALL) INDEX(ALL) SHRLEVEL CHANGEUpdates system catalog statistics online.
118Run Check Data UtilityCHECK DATA TABLESPACE PRODDB.TS01 FOR EXCEPTIONValidates referential integrity constraint rules.
119Run Check Index UtilityCHECK INDEX (ALL) TABLESPACE PRODDB.TS01Scans index pages for structural corruption.
120Run Repair Set UtilityREPAIR SET TABLESPACE PRODDB.TS01 NOCHECKPENDResets check pending status flag.
121Run Quiesce Table Space UtilityQUIESCE TABLESPACE PRODDB.TS01Enforces write flush and records log LSN point.
122Run Modify Recovery UtilityMODIFY RECOVERY TABLESPACE PRODDB.TS01 DELETE AGE(60)Purges image copy records older than 60 days.
123Display RLST Table-DISPLAY RLSTInspects active Resource Limit Specification Table ID.
124Start RLST Table-START RLST ID=01Activates SQL CPU governor rules from RLST 01.
125Stop RLST Table-STOP RLSTDeactivates SQL CPU governor rules.
126Bind Package ReplaceBIND PACKAGE(PRODCOLL) MEMBER(PRODPROG) ACTION(REPLACE) ISOLATION(CS)Binds static package with Cursor Stability.
127Free Package VersionFREE PACKAGE(PRODCOLL.PRODPROG.(V1))Deletes static package version from catalog.
128Rebind Package VersionREBIND PACKAGE(PRODCOLL.PRODPROG)Recompiles static package execution paths.
129Check Catalog Tablespace PagesSELECT NAME, NACTIVE FROM SYSIBM.SYSTABLESPACE WHERE DBNAME='PRODDB';Inspects total allocated pages in tablespace.
130Check System Subsystem VersionSELECT VERSION FROM SYSIBM.SYSENVIRONMENT;Outputs Db2 z/OS subsystem release and version string.
131Display Storage Group Detail-DISPLAY GROUPInspects data sharing group subsystem state.
132Check Catalog Database ListSELECT NAME, CREATOR, STOGROUP FROM SYSIBM.SYSDATABASES;Lists all databases defined in catalog.
133Check Catalog Tablespace DetailSELECT NAME, DBNAME, PAGESIZE, BUFFERPOOL FROM SYSIBM.SYSTABLESPACE;Inspects page size and buffer pool for tablespaces.
134Check Catalog Tables CountSELECT COUNT(*) FROM SYSIBM.SYSTABLES;Counts total tables defined in system catalog.
135Check Catalog Indexes CountSELECT COUNT(*) FROM SYSIBM.SYSINDEXES;Counts total indexes defined in system catalog.
136Check Catalog Foreign KeysSELECT RELNAME, TBNAME, REFTBNAME FROM SYSIBM.SYSRELS;Lists foreign key relationships in system catalog.
137Check Catalog Views ListSELECT NAME, CREATOR FROM SYSIBM.SYSVIEWS;Lists views defined in system catalog.
138Check Catalog Triggers ListSELECT NAME, SCHEMA, TBNAME FROM SYSIBM.SYSTRIGGERS;Lists triggers defined in system catalog.
139Check Catalog Routines ListSELECT NAME, SCHEMA, ROUTINETYPE FROM SYSIBM.SYSROUTINES;Lists stored procedures and UDFs in system catalog.
140Check Catalog Synonyms ListSELECT NAME, TBNAME FROM SYSIBM.SYSSYNONYMS;Lists synonyms defined in system catalog.
141Check Catalog User AuthorizationsSELECT GRANTEE, AUTHHOWGOT FROM SYSIBM.SYSUSERAUTH;Lists system authorization privileges assigned to users.
142Check Catalog DB AuthorizationsSELECT GRANTEE, DBNAME, DBADMAUTH FROM SYSIBM.SYSDBAUTH;Lists database administrative privileges.
143Check Catalog Table AuthorizationsSELECT GRANTEE, TBNAME, SELECTAUTH FROM SYSIBM.TABAUTH;Lists table read privileges in system catalog.
144Check Catalog Package AuthorizationsSELECT GRANTEE, PACKNAME, EXECUTEAUTH FROM SYSIBM.SYSPACKAUTH;Lists package execution privileges in system catalog.
145Check Catalog Plan AuthorizationsSELECT GRANTEE, PLANNAME, EXECUTEAUTH FROM SYSIBM.SYSPLANAUTH;Lists plan execution privileges in system catalog.
146Check Catalog Buffer Pool ListSELECT BPNAME, VPSIZE FROM SYSIBM.SYSBUFFERPOOLS;Lists buffer pool configurations.
147Check Catalog Storage Group ListSELECT NAME, VOLID FROM SYSIBM.SYSVOLUMES;Lists DASD volume serials in storage groups.
148Check Catalog Image Copy ListSELECT DBNAME, TSNAME, DATE, START_LBN FROM SYSIBM.SYSCOPY ORDER BY DATE DESC;Lists historical image copies in catalog.
149Check Catalog Utility StatusSELECT UTILID, STATE FROM SYSIBM.SYSUTIL;Lists active utility execution states in catalog.
150Check Catalog Pending StatusSELECT DBNAME, NAME, STAT TYPE FROM SYSIBM.SYSPENDING;Lists tablespaces in advisory pending status.
151Check Catalog LOB Table SpacesSELECT NAME, DBNAME FROM SYSIBM.SYSTABLESPACE WHERE TYPE='LOB';Lists LOB tablespaces in system catalog.
152Check Catalog XML Table SpacesSELECT NAME, DBNAME FROM SYSIBM.SYSTABLESPACE WHERE TYPE='XML';Lists XML tablespaces in system catalog.
153Check Catalog Partition StorageSELECT DBNAME, TSNAME, PARTITION, PQTY, SQTY FROM SYSIBM.SYSTABLEPART;Inspects primary/secondary VSAM allocation quantities.
154Check Catalog Index Partition StorageSELECT IXNAME, PARTITION, PQTY, SQTY FROM SYSIBM.SYSINDEXPART;Inspects index partition VSAM allocation quantities.
155Check Catalog Plan StatementsSELECT PLANNAME, STMTNO, TEXT FROM SYSIBM.SYSSTMT;Inspects static SQL statement text bound to plans.
156Check Catalog Package StatementsSELECT PKGNAME, STMTNO, TEXT FROM SYSIBM.SYSPACKSTMT;Inspects static SQL statement text bound to packages.
157Check Catalog DDF LocationsSELECT LINKNAME, IPADDR, PORT FROM SYSIBM.SYSLUNAMES;Inspects remote location network addresses in catalog.
158Check Catalog DDF User MappingsSELECT USERNM, RECORGNM FROM SYSIBM.SYSUSERNAMES;Inspects inbound remote user authentication mappings.
159Check Catalog System ParametersSELECT ZPARM_NAME, ZPARM_VALUE FROM SYSIBM.SYSZPARMS;Inspects system parameter overrides.
160Check Db2 Subsystem NameSELECT SUBSYSTEM_NAME FROM SYSIBM.SYSENVIRONMENT;Outputs active Db2 subsystem name.
161Check Db2 Workload Manager ClassSELECT WLM_ENV FROM SYSIBM.SYSROUTINES WHERE WLM_ENV IS NOT NULL;Lists z/OS WLM environment names for stored procedures.
162Check Db2 Coupling Facility LocksSELECT * FROM SYSIBM.SYSLOCKS WHERE LOCK_TYPE='GBP';Inspects group buffer pool coupling facility locks.
163Check Db2 Log Buffer SizeSELECT VALUE FROM SYSIBM.SYSZPARMS WHERE ZPARM_NAME='OUTBUFF';Inspects output log buffer RAM size.
164Check Db2 Max Remote ConnectionsSELECT VALUE FROM SYSIBM.SYSZPARMS WHERE ZPARM_NAME='CONDBAT';Inspects max concurrent remote DDF threads.
165Check Db2 Max Active UsersSELECT VALUE FROM SYSIBM.SYSZPARMS WHERE ZPARM_NAME='CTHREAD';Inspects max concurrent allied threads.
166Check Db2 Lock Timeout ParameterSELECT VALUE FROM SYSIBM.SYSZPARMS WHERE ZPARM_NAME='IRLMRLT';Inspects IRLM lock wait timeout seconds.
167Check Db2 Deadlock Timeout ParameterSELECT VALUE FROM SYSIBM.SYSZPARMS WHERE ZPARM_NAME='DEADLOCK';Inspects IRLM deadlock detection interval milliseconds.
168Check Db2 Checkpoint Frequency ParameterSELECT VALUE FROM SYSIBM.SYSZPARMS WHERE ZPARM_NAME='CHKFREQ';Inspects checkpoint frequency in log records or minutes.
169Check Db2 Max Storage Files Per SpaceSELECT VALUE FROM SYSIBM.SYSZPARMS WHERE ZPARM_NAME='MAXCAP';Inspects maximum VSAM datasets per tablespace.
170Check Db2 Archive Log Destination 1SELECT VALUE FROM SYSIBM.SYSZPARMS WHERE ZPARM_NAME='ARCLOG1';Inspects primary archive log dataset prefix.
171Check Db2 Archive Log Destination 2SELECT VALUE FROM SYSIBM.SYSZPARMS WHERE ZPARM_NAME='ARCLOG2';Inspects secondary archive log dataset prefix.
172Check Db2 System Uptime TimestampSELECT START_TIME FROM SYSIBM.SYSENVIRONMENT;Outputs subsystem startup timestamp.
173Check Db2 Query PingSELECT 1 FROM SYSIBM.SYSDUMMY1;Simple connection ping query against SYSDUMMY1.
174Check Db2 Current DateSELECT CURRENT DATE FROM SYSIBM.SYSDUMMY1;Returns current z/OS system date.
175Check Db2 Current TimeSELECT CURRENT TIME FROM SYSIBM.SYSDUMMY1;Returns current z/OS system time.
176Check Db2 Current TimestampSELECT CURRENT TIMESTAMP FROM SYSIBM.SYSDUMMY1;Returns current z/OS system timestamp.
177Check Db2 Current UserSELECT USER FROM SYSIBM.SYSDUMMY1;Returns active execution user primary auth ID.
178Check Db2 Current SQLIDSELECT CURRENT SQLID FROM SYSIBM.SYSDUMMY1;Returns active current SQLID for DDL.
179Check Db2 Current Server NameSELECT CURRENT SERVER FROM SYSIBM.SYSDUMMY1;Returns active subsystem location name.
180Check Db2 Current TimezoneSELECT CURRENT TIMEZONE FROM SYSIBM.SYSDUMMY1;Returns active system timezone offset.
181Check Db2 Current SchemaSELECT CURRENT SCHEMA FROM SYSIBM.SYSDUMMY1;Returns active default schema name.
182Check Db2 Current Package VersionSELECT CURRENT PACKAGE VERSION FROM SYSIBM.SYSDUMMY1;Returns active running package version string.
183Check Db2 Current PrecisionSELECT CURRENT PRECISION FROM SYSIBM.SYSDUMMY1;Returns active decimal math precision setting.
184Check Db2 Current RulesSELECT CURRENT RULES FROM SYSIBM.SYSDUMMY1;Returns active SQL rules mode (DB2 vs STD).
185Check Db2 Current DegreeSELECT CURRENT DEGREE FROM SYSIBM.SYSDUMMY1;Returns active query parallelism degree.
186Check Db2 Current Application EncodingSELECT CURRENT APPLICATION ENCODING SCHEME FROM SYSIBM.SYSDUMMY1;Returns active EBCDIC or ASCII encoding scheme.
187Check Db2 Subsystem Build LevelSELECT GETVARIABLE('SYSIBM.VERSION') FROM SYSIBM.SYSDUMMY1;Outputs exact Db2 z/OS version string (DSN13010).
188Check Db2 Subsystem Code PageSELECT GETVARIABLE('SYSIBM.SYSTEM_EBCDIC_CCSID') FROM SYSIBM.SYSDUMMY1;Outputs system EBCDIC code page number.
189Check Db2 Subsystem Timezone NameSELECT GETVARIABLE('SYSIBM.TIMEZONE_NAME') FROM SYSIBM.SYSDUMMY1;Outputs system IANA timezone name.
190Check Db2 Subsystem Host NameSELECT GETVARIABLE('SYSIBM.HOSTNAME') FROM SYSIBM.SYSDUMMY1;Outputs mainframe z/OS LPAR host name.
191Check Db2 Subsystem IP AddressSELECT GETVARIABLE('SYSIBM.IPADDR') FROM SYSIBM.SYSDUMMY1;Outputs primary IP address of mainframe DDF listener.
192Check Db2 Subsystem Job NameSELECT GETVARIABLE('SYSIBM.JOB_NAME') FROM SYSIBM.SYSDUMMY1;Outputs active batch job name or DSN1MSTR task name.
193Check Db2 Subsystem Job IDSELECT GETVARIABLE('SYSIBM.JOB_ID') FROM SYSIBM.SYSDUMMY1;Outputs active JES2 job ID.
194Check Db2 Subsystem Step NameSELECT GETVARIABLE('SYSIBM.STEP_NAME') FROM SYSIBM.SYSDUMMY1;Outputs active job step name.
195Check Db2 Subsystem User NameSELECT GETVARIABLE('SYSIBM.USER_NAME') FROM SYSIBM.SYSDUMMY1;Outputs authenticated RACF/ACF2 user ID.
196Check Db2 Subsystem Client IPSELECT GETVARIABLE('SYSIBM.CLIENT_IPADDR') FROM SYSIBM.SYSDUMMY1;Outputs remote client IP address.
197Check Db2 Subsystem Client App NameSELECT GETVARIABLE('SYSIBM.CLIENT_APPLNAME') FROM SYSIBM.SYSDUMMY1;Outputs remote client application name.
198Check Db2 Subsystem Client User NameSELECT GETVARIABLE('SYSIBM.CLIENT_USERID') FROM SYSIBM.SYSDUMMY1;Outputs remote client user ID.
199Check Db2 Subsystem Client Workstation NameSELECT GETVARIABLE('SYSIBM.CLIENT_WRKSTNNAME') FROM SYSIBM.SYSDUMMY1;Outputs remote workstation host name.
200Verify Db2 Ping ConnectionSELECT 1 FROM SYSIBM.SYSDUMMY1;Verifies active connection to Db2 subsystem.

MariaDB OLTP

Technical Architecture & Overview

MariaDB is an open-source relational database created by the original developers of MySQL. It expands traditional relational capabilities with modern pluggable storage engines (InnoDB, Aria, ColumnStore, MyRocks, Spider), temporal system-versioning tables, sequence objects, and Galera multi-master synchronous clustering.

Primary Use Cases: Web application transactions, high-concurrency OLTP, columnar real-time analytics (ColumnStore), temporal auditing, and multi-master scale-out clusters.

Core Components: Server Daemon (mariadbd), Aria Engine, InnoDB Buffer Pool, Galera Replication Plugin (wsrep), ColumnStore Analytics Engine, and Binary Logs.

Exhaustive 200-command administrative, diagnostic, and operational matrix for MariaDB.

#Operation / ActionSQL / CLI SyntaxDescription
1Start MariaDB Daemonsudo systemctl start mariadbBoots MariaDB server service.
2Stop MariaDB Daemonsudo systemctl stop mariadbStops MariaDB server service.
3Interactive Consolemariadb -u root -pOpens administrative SQL monitor.
4Logical Backupmariadb-dump -u root -p --all-databases > all_dbs.sqlExports database cluster dump.
5Restore Dumpmariadb -u root -p enterprise_db < all_dbs.sqlRestores SQL backup dump.
6Check Aria Engine StatusSHOW ENGINE ARIARUNNING STATUS;Inspects Aria storage engine metrics.
7Inspect Active ProcessesSHOW PROCESSLIST;Views running client connections.
8Kill Query ThreadKILL QUERY 832;Terminates long-running query.
9Create DatabaseCREATE DATABASE enterprise_db CHARACTER SET utf8mb4;Provisions new database.
10Drop DatabaseDROP DATABASE enterprise_db;Deletes database instance.
11Create UserCREATE USER 'app_user'@'localhost' IDENTIFIED BY 'SecurePass!';Provisions database user.
12Grant Access PrivilegesGRANT ALL PRIVILEGES ON enterprise_db.* TO 'app_user'@'localhost';Assigns database rights.
13Analyze TableANALYZE TABLE users;Refreshes index statistics.
14Optimize TableOPTIMIZE TABLE users;Defragments Aria/InnoDB storage.
15Check Table IntegrityCHECK TABLE users EXTENDED;Performs deep table corruption scan.
16Repair TableREPAIR TABLE users;Repairs corrupted table file.
17Explain Query PlanEXPLAIN SELECT * FROM users WHERE status = 'active';Evaluates query execution strategy.
18Show Status VariablesSHOW GLOBAL STATUS LIKE 'Conn%';Inspects connection counters.
19Show System VariablesSHOW VARIABLES LIKE 'innodb_buffer_pool_size';Inspects configuration parameters.
20Set Runtime VariableSET GLOBAL max_allowed_packet = 67108864;Modifies network buffer limit.
21Check Replication StatusSHOW ALL SLAVES STATUS;Inspects multi-source replication sync.
22Reset ReplicaRESET SLAVE;Clears replication configurations.
23Change Replication MasterCHANGE MASTER TO MASTER_HOST='10.0.0.10';Sets replication upstream source.
24Create IndexCREATE INDEX idx_user_status ON users(status);Provisions new table index.
25Drop IndexDROP INDEX idx_user_status ON users;Removes table index.
26Flush Table PrivilegeFLUSH PRIVILEGES;Reloads grant permission tables.
27Show Binary LogsSHOW BINARY LOGS;Lists active transaction binlogs.
28Purge Binary LogsPURGE BINARY LOGS TO 'mariadb-bin.000450';Removes old transaction binlogs.
29Check Disk UsageSELECT table_name, data_length FROM information_schema.tables WHERE table_schema='enterprise_db';Calculates table storage sizes.
30Version InspectionSELECT VERSION();Outputs running MariaDB engine version.
31Start Slave ThreadSTART SLAVE;Starts replication receiver and SQL worker threads.
32Stop Slave ThreadSTOP SLAVE;Stops replication receiver and SQL worker threads.
33Start Named SlaveSTART SLAVE 'master_node2';Starts specific multi-source replication channel.
34Stop Named SlaveSTOP SLAVE 'master_node2';Stops specific multi-source replication channel.
35Show Named Slave StatusSHOW SLAVE 'master_node2' STATUS\GInspects status of named replication channel.
36Check Galera Cluster StatusSHOW STATUS LIKE 'wsrep_cluster_size';Inspects number of active nodes in Galera cluster.
37Check Galera Sync StateSHOW STATUS LIKE 'wsrep_local_state_comment';Verifies if node is Synced in Galera cluster.
38Check Galera Flow ControlSHOW STATUS LIKE 'wsrep_flow_control_paused';Monitors Galera replication flow control pause time.
39Set Galera Desync ModeSET GLOBAL wsrep_desync = ON;Desynchronizes node from cluster for maintenance.
40Re-sync Galera NodeSET GLOBAL wsrep_desync = OFF;Re-synchronizes node with Galera cluster.
41Check ColumnStore StatusSELECT * FROM information_schema.columnstore_tables;Inspects MariaDB ColumnStore engine tables.
42Create ColumnStore TableCREATE TABLE sales_fact (id INT, amount DECIMAL(10,2)) ENGINE=ColumnStore;Provisions analytical columnar table.
43Create SequenceCREATE SEQUENCE user_seq START WITH 1000 INCREMENT BY 1;Provisions database sequence object.
44Get Sequence ValueSELECT NEXT VALUE FOR user_seq;Fetches next value from sequence.
45Drop SequenceDROP SEQUENCE user_seq;Deletes sequence object.
46Create Flashback TableCREATE TABLE users_history FOR SYSTEM_TIME ALL SELECT * FROM users;Inspects temporal table history.
47Query System Time TravelSELECT * FROM users FOR SYSTEM_TIME AS OF '2026-07-01 10:00:00';Queries historical state via system-versioning.
48Create System-Versioned TableCREATE TABLE users (id INT, name VARCHAR(50)) WITH SYSTEM VERSIONING;Provisions temporal historical table.
49Set Password ExpiryALTER USER 'app_user'@'localhost' PASSWORD EXPIRE INTERVAL 90 DAY;Enforces password rotation policy.
50Lock User AccountALTER USER 'app_user'@'localhost' ACCOUNT LOCK;Disables user authentication.
51Unlock User AccountALTER USER 'app_user'@'localhost' ACCOUNT UNLOCK;Enables user authentication.
52Show User GrantsSHOW GRANTS FOR 'app_user'@'localhost';Lists all assigned permissions for user.
53Revoke PrivilegesREVOKE ALL PRIVILEGES ON enterprise_db.* FROM 'app_user'@'localhost';Removes permissions from user.
54Create RoleCREATE ROLE 'app_read_role';Provisions security role container.
55Grant Role to UserGRANT 'app_read_role' TO 'app_user'@'localhost';Assigns role to user account.
56Set Default RoleSET DEFAULT ROLE 'app_read_role' FOR 'app_user'@'localhost';Activates default role on login.
57Drop RoleDROP ROLE 'app_read_role';Deletes security role container.
58Show EnginesSHOW ENGINES;Lists supported storage engines and default state.
59Check Aria Log FileSHOW ENGINE ARIA STATUS;Inspects Aria recovery log state.
60Set InnoDB Buffer Pool SizeSET GLOBAL innodb_buffer_pool_size = 4294967296;Adjusts RAM assigned to buffer pool.
61Set Slow Query LogSET GLOBAL slow_query_log = 1;Activates slow query logging.
62Set Long Query TimeSET GLOBAL long_query_time = 1.5;Logs queries taking over 1.5 seconds.
63Set Max ConnectionsSET GLOBAL max_connections = 1000;Increases max client connection limit.
64Set Read Only ModeSET GLOBAL read_only = ON;Blocks writes for non-admin accounts.
65Disable Read Only ModeSET GLOBAL read_only = OFF;Enables database write operations.
66Create Range Partitioned TableCREATE TABLE orders (id INT, dt DATE) PARTITION BY RANGE (YEAR(dt)) (PARTITION p0 VALUES LESS THAN (2026));Provisions range partition table.
67Add PartitionALTER TABLE orders ADD PARTITION (PARTITION p1 VALUES LESS THAN (2027));Appends new partition range.
68Drop PartitionALTER TABLE orders DROP PARTITION p0;Deletes partition data instantly.
69Truncate PartitionALTER TABLE orders TRUNCATE PARTITION p1;Clears partition records.
70Check Partition InfoSELECT PARTITION_NAME, TABLE_ROWS FROM information_schema.PARTITIONS WHERE TABLE_NAME='orders';Inspects table partition metrics.
71Create Virtual ColumnALTER TABLE users ADD COLUMN full_name VARCHAR(100) AS (CONCAT(first_name, ' ', last_name)) VIRTUAL;Provisions computed virtual column.
72Create Stored ColumnALTER TABLE users ADD COLUMN full_name VARCHAR(100) AS (CONCAT(first_name, ' ', last_name)) STORED;Provisions computed persistent column.
73Create Fulltext IndexCREATE FULLTEXT INDEX idx_ft_bio ON users(bio);Provisions inverted index for text search.
74Run Fulltext SearchSELECT * FROM users WHERE MATCH(bio) AGAINST('engineer' IN BOOLEAN MODE);Executes fulltext search query.
75Create Invisible IndexALTER TABLE users ADD INDEX idx_status (status) INVISIBLE;Provisions index hidden from optimizer.
76Make Index VisibleALTER TABLE users ALTER INDEX idx_status VISIBLE;Restores index visibility to optimizer.
77Show Create TableSHOW CREATE TABLE users;Outputs exact DDL syntax used to build table.
78Show Create DatabaseSHOW CREATE DATABASE enterprise_db;Outputs exact DDL syntax used to build DB.
79Show Create ViewSHOW CREATE VIEW v_users;Outputs exact DDL syntax used to build view.
80Create TriggerCREATE TRIGGER trg_user_bi BEFORE INSERT ON users FOR EACH ROW SET NEW.created_at = NOW();Provisions pre-insert trigger.
81Drop TriggerDROP TRIGGER IF EXISTS trg_user_bi;Deletes trigger routine.
82Show TriggersSHOW TRIGGERS FROM enterprise_db;Lists all triggers in database.
83Create Event SchedulerCREATE EVENT ev_purge ON SCHEDULE EVERY 1 DAY DO DELETE FROM logs WHERE dt < NOW() - INTERVAL 7 DAY;Provisions automated event.
84Show Scheduled EventsSHOW EVENTS;Lists all configured scheduled events.
85Drop Scheduled EventDROP EVENT IF EXISTS ev_purge;Deletes scheduled event.
86Check Table Lock WaitersSELECT * FROM information_schema.INNODB_LOCK_WAITS;Inspects blocking lock wait chains.
87Check Table Locks HeldSELECT * FROM information_schema.INNODB_LOCKS;Inspects active InnoDB locks.
88Check Transaction StatusSELECT * FROM information_schema.INNODB_TRX;Inspects active transactions.
89Create SavepointSAVEPOINT svp_1;Sets savepoint inside transaction.
90Rollback to SavepointROLLBACK TO SAVEPOINT svp_1;Reverts transaction state to savepoint.
91Release SavepointRELEASE SAVEPOINT svp_1;Removes savepoint reference.
92Check Max Allowed PacketSELECT @@max_allowed_packet;Inspects max network packet size threshold.
93Check Character SetsSHOW CHARACTER SET;Outputs supported character set encodings.
94Check CollationsSHOW COLLATION WHERE Charset='utf8mb4';Lists supported utf8mb4 collation rules.
95Check UptimeSHOW GLOBAL STATUS LIKE 'Uptime';Displays server uptime in seconds.
96Flush Query CacheFLUSH QUERY CACHE;Defragments query cache memory.
97Reset Query CacheRESET QUERY CACHE;Clears query cache memory contents.
98Flush User ResourcesFLUSH USER_RESOURCES;Resets user resource limit counters.
99Flush Error LogsFLUSH ERROR LOGS;Rotates error log files.
100Shutdown Server via SQLSHUTDOWN;Triggers graceful server shutdown via SQL.
101Check Galera Cluster Size StatusSHOW STATUS LIKE 'wsrep_cluster_size';Inspects active node count in Galera cluster.
102Check Galera Node Sync StatusSHOW STATUS LIKE 'wsrep_local_state_comment';Verifies if node is Synced in Galera cluster.
103Check Galera Flow Control StatusSHOW STATUS LIKE 'wsrep_flow_control_paused';Monitors replication flow control pause time.
104Set Galera Desync ModeSET GLOBAL wsrep_desync = ON;Desynchronizes node for maintenance operations.
105Set Galera Resync ModeSET GLOBAL wsrep_desync = OFF;Re-synchronizes node with Galera cluster.
106Check ColumnStore TablesSELECT * FROM information_schema.columnstore_tables;Inspects MariaDB ColumnStore engine tables.
107Create ColumnStore TableCREATE TABLE sales_col (id INT, amount DECIMAL(10,2)) ENGINE=ColumnStore;Provisions analytical columnar table.
108Create Sequence ObjectCREATE SEQUENCE user_seq START WITH 1000 INCREMENT BY 1;Provisions database sequence object.
109Get Sequence Next ValueSELECT NEXT VALUE FOR user_seq;Fetches next value from sequence.
110Drop Sequence ObjectDROP SEQUENCE user_seq;Deletes sequence object.
111Create System-Versioned TableCREATE TABLE users (id INT, name VARCHAR(50)) WITH SYSTEM VERSIONING;Provisions temporal historical table.
112Query System-Versioned TableSELECT * FROM users FOR SYSTEM_TIME AS OF '2026-07-01 10:00:00';Queries historical state via system-versioning.
113Check Aria Engine StatusSHOW ENGINE ARIA STATUS;Inspects Aria crash-safe storage engine state.
114Set Password Expiry IntervalALTER USER 'app_user'@'localhost' PASSWORD EXPIRE INTERVAL 90 DAY;Enforces password rotation policy.
115Lock Account LoginALTER USER 'app_user'@'localhost' ACCOUNT LOCK;Disables user authentication.
116Unlock Account LoginALTER USER 'app_user'@'localhost' ACCOUNT UNLOCK;Enables user authentication.
117Show User GrantsSHOW GRANTS FOR 'app_user'@'localhost';Lists permissions assigned to user.
118Create RoleCREATE ROLE 'app_read_role';Provisions security role container.
119Grant Role to UserGRANT 'app_read_role' TO 'app_user'@'localhost';Assigns role to user account.
120Drop RoleDROP ROLE 'app_read_role';Deletes security role container.
121Set InnoDB Buffer Pool SizeSET GLOBAL innodb_buffer_pool_size = 4294967296;Adjusts RAM assigned to buffer pool.
122Set Slow Query LogSET GLOBAL slow_query_log = 1;Activates slow query logging.
123Set Long Query TimeSET GLOBAL long_query_time = 1.5;Logs queries taking over 1.5 seconds.
124Set Max ConnectionsSET GLOBAL max_connections = 1000;Increases max client connection limit.
125Set Read Only ModeSET GLOBAL read_only = ON;Blocks writes for non-admin accounts.
126Disable Read Only ModeSET GLOBAL read_only = OFF;Enables database write operations.
127Check Table Lock WaitersSELECT * FROM information_schema.INNODB_LOCK_WAITS;Inspects blocking lock wait chains.
128Check Active TransactionsSELECT * FROM information_schema.INNODB_TRX;Inspects active InnoDB transactions.
129Verify Engine VersionSELECT VERSION();Outputs running MariaDB engine version string.
130Shutdown MariaDB ServerSHUTDOWN;Gracefully shuts down MariaDB server daemon.
131Check Galera Cluster StateSHOW STATUS LIKE 'wsrep_local_state';Inspects numeric state of local Galera node.
132Check Galera Cluster Status CommentSHOW STATUS LIKE 'wsrep_local_state_comment';Inspects human-readable state (Synced/Donor/Joined).
133Check Galera Cluster Conf IDSHOW STATUS LIKE 'wsrep_cluster_conf_id';Inspects cluster configuration change counter.
134Check Galera Cluster WeightSHOW STATUS LIKE 'wsrep_cluster_weight';Inspects total quorum weight of active cluster.
135Check Galera Cluster Status HealthSHOW STATUS LIKE 'wsrep_connected';Verifies whether node is connected to Galera cluster.
136Check Galera Ready StatusSHOW STATUS LIKE 'wsrep_ready';Verifies whether node is ready to accept SQL queries.
137Check Galera Received QueueSHOW STATUS LIKE 'wsrep_local_recv_queue';Inspects length of received replication queue.
138Check Galera Send QueueSHOW STATUS LIKE 'wsrep_local_send_queue';Inspects length of local send queue.
139Check Galera Cert FailuresSHOW STATUS LIKE 'wsrep_local_cert_failures';Monitors total certification conflict transaction aborts.
140Check Galera BF AbortsSHOW STATUS LIKE 'wsrep_local_bf_aborts';Monitors brute-force transaction aborts by high priority cluster node.
141Check Aria Page Cache SizeSHOW VARIABLES LIKE 'aria_pagecache_buffer_size';Inspects RAM allocated for Aria storage engine page cache.
142Set Aria Page Cache SizeSET GLOBAL aria_pagecache_buffer_size = 1073741824;Allocates 1GB RAM for Aria page cache.
143Check Aria Log File SizeSHOW VARIABLES LIKE 'aria_log_file_size';Inspects max size of Aria transaction log files.
144Check Aria Recover ModeSHOW VARIABLES LIKE 'aria_recover_options';Inspects automatic recovery mode for corrupted Aria tables.
145Check ColumnStore Engine VersionSELECT * FROM information_schema.plugins WHERE plugin_name='ColumnStore';Verifies ColumnStore plugin release version.
146Check Sequence Cache SizeCREATE SEQUENCE seq_cached START 1 CACHE 1000;Provisions sequence with 1000 in-memory pre-allocated IDs.
147Check System Versioning ModeSHOW VARIABLES LIKE 'system_versioning_asof';Inspects default system-versioning time travel mode.
148Alter System Versioned Table Add ColumnALTER TABLE users ADD COLUMN bio TEXT;Appends column preserving historical temporal versions.
149Alter System Versioned Table Drop VersioningALTER TABLE users DROP SYSTEM VERSIONING;Removes system-versioning tracking from table.
150Check InnoDB Buffer Pool InstancesSHOW VARIABLES LIKE 'innodb_buffer_pool_instances';Inspects buffer pool shard count.
151Check InnoDB Log File SizeSHOW VARIABLES LIKE 'innodb_log_file_size';Inspects size of redo log files.
152Check Max Allowed PacketSHOW VARIABLES LIKE 'max_allowed_packet';Inspects maximum allowed packet payload size.
153Set Max Allowed PacketSET GLOBAL max_allowed_packet = 1073741824;Increases max packet payload size to 1GB.
154Check Slow Query Log LocationSHOW VARIABLES LIKE 'slow_query_log_file';Inspects file path of slow query log.
155Check Error Log LocationSHOW VARIABLES LIKE 'log_error';Inspects file path of server error log.
156Check Query Exec Time LimitSET SESSION max_statement_time = 10.0;Caps SQL query execution time to 10 seconds.
157Check Active Server ThreadsSHOW GLOBAL STATUS LIKE 'Threads_running';Monitors active concurrent query execution threads.
158Check Server Connection LimitsSHOW VARIABLES LIKE 'max_connections';Inspects maximum allowed client connections.
159Check Server Uptime StatusSHOW GLOBAL STATUS LIKE 'Uptime';Displays server uptime in seconds.
160Check Database Ping QuerySELECT 1;Simple connection ping test returning 1.
161Check Current User AccountSELECT CURRENT_USER();Returns current execution user account.
162Check Current Database NameSELECT DATABASE();Returns active selected database name.
163Check System TimestampSELECT NOW();Returns current server timestamp.
164Check System UTC TimestampSELECT UTC_TIMESTAMP();Returns current UTC timestamp.
165Check Connection Thread IDSELECT CONNECTION_ID();Returns current connection thread ID.
166Check MariaDB Version DetailsSELECT VERSION();Outputs MariaDB server build and patch version string.
167Check Server Host NameSHOW VARIABLES LIKE 'hostname';Outputs host server hostname.
168Check Server Port NumberSHOW VARIABLES LIKE 'port';Outputs TCP/IP listening port number (3306).
169Check Socket File PathSHOW VARIABLES LIKE 'socket';Outputs Unix domain socket file path.
170Check Data Directory PathSHOW VARIABLES LIKE 'datadir';Outputs physical database files storage path.
171Check Tmp Directory PathSHOW VARIABLES LIKE 'tmpdir';Outputs temporary files storage path.
172Check Open File LimitSHOW VARIABLES LIKE 'open_files_limit';Outputs max OS file handles allowed for server.
173Check Key Buffer Size (MyISAM)SHOW VARIABLES LIKE 'key_buffer_size';Inspects memory allocated for MyISAM index blocks.
174Check Read Buffer SizeSHOW VARIABLES LIKE 'read_buffer_size';Inspects per-thread sequential scan buffer size.
175Check Read Rnd Buffer SizeSHOW VARIABLES LIKE 'read_rnd_buffer_size';Inspects per-thread sorted scan buffer size.
176Check Sort Buffer SizeSHOW VARIABLES LIKE 'sort_buffer_size';Inspects per-thread ORDER BY sort buffer size.
177Check Join Buffer SizeSHOW VARIABLES LIKE 'join_buffer_size';Inspects per-thread full join buffer size.
178Check Max Heap Table SizeSHOW VARIABLES LIKE 'max_heap_table_size';Inspects max size for MEMORY engine tables.
179Check Tmp Table SizeSHOW VARIABLES LIKE 'tmp_table_size';Inspects max size for internal in-memory temp tables.
180Check Created Tmp Disk TablesSHOW GLOBAL STATUS LIKE 'Created_tmp_disk_tables';Monitors internal temp tables spilling to disk.
181Check Created Tmp TablesSHOW GLOBAL STATUS LIKE 'Created_tmp_tables';Monitors total internal temp tables created.
182Check Select Scan CountSHOW GLOBAL STATUS LIKE 'Select_scan';Monitors full table scan execution count.
183Check Select Full Join CountSHOW GLOBAL STATUS LIKE 'Select_full_join';Monitors joins missing index keys.
184Check Slow Queries CountSHOW GLOBAL STATUS LIKE 'Slow_queries';Monitors total slow queries executed.
185Check Com Select CountSHOW GLOBAL STATUS LIKE 'Com_select';Monitors SELECT statements executed.
186Check Com Insert CountSHOW GLOBAL STATUS LIKE 'Com_insert';Monitors INSERT statements executed.
187Check Com Update CountSHOW GLOBAL STATUS LIKE 'Com_update';Monitors UPDATE statements executed.
188Check Com Delete CountSHOW GLOBAL STATUS LIKE 'Com_delete';Monitors DELETE statements executed.
189Check Connections CountSHOW GLOBAL STATUS LIKE 'Connections';Monitors total connection attempts since boot.
190Check Aborted ConnectsSHOW GLOBAL STATUS LIKE 'Aborted_connects';Monitors failed client connection attempts.
191Check Aborted ClientsSHOW GLOBAL STATUS LIKE 'Aborted_clients';Monitors client connections dropped improperly.
192Check Character Set SystemSHOW VARIABLES LIKE 'character_set_system';Inspects system metadata character set (utf8mb3).
193Check Character Set ServerSHOW VARIABLES LIKE 'character_set_server';Inspects default server character set.
194Check Collation ServerSHOW VARIABLES LIKE 'collation_server';Inspects default server collation.
195Check Time Zone VariableSHOW VARIABLES LIKE 'time_zone';Inspects active server time zone.
196Set Time Zone SessionSET time_zone = '+00:00';Configures active session time zone to UTC.
197Check Default Storage EngineSHOW VARIABLES LIKE 'default_storage_engine';Inspects default storage engine (InnoDB).
198Check SQL ModeSHOW VARIABLES LIKE 'sql_mode';Inspects active SQL syntax mode and strictness flags.
199Set SQL Mode StrictSET GLOBAL sql_mode = 'STRICT_TRANS_TABLES,ERROR_FOR_DIVISION_BY_ZERO,NO_AUTO_CREATE_USER';Enforces strict transaction safety mode.
200Shutdown MariaDB Server SQLSHUTDOWN;Gracefully shuts down MariaDB server daemon.

SQLite OLTP

Technical Architecture & Overview

SQLite is a C-language library that implements a self-contained, serverless, zero-configuration, transactional SQL database engine. It writes directly to a single disk file, featuring Write-Ahead Logging (WAL) concurrency, full-text search (FTS5), and R-Tree spatial extensions.

Primary Use Cases: Embedded systems, mobile applications (iOS/Android), desktop software, edge IoT devices, local application caching, and lightweight standalone testing environments.

Core Components: SQL Compiler, Virtual Machine (VDBE), B-Tree Manager, Page Cache, OS Interface (VFS), and WAL Journal File.

Exhaustive 200-command administrative, diagnostic, and operational matrix for SQLite.

#Operation / ActionCLI / PRAGMA SyntaxDescription
1Open Database CLIsqlite3 enterprise.dbLaunches interactive SQLite shell.
2Backup Database Filesqlite3 enterprise.db ".backup 'backup.db'"Creates a safe online database backup.
3Restore Databasesqlite3 enterprise.db ".restore 'backup.db'"Restores database from backup file.
4Export SQL Dumpsqlite3 enterprise.db ".dump" > dump.sqlExports schema and records to SQL.
5Import SQL Dumpsqlite3 enterprise.db ".read dump.sql"Imports SQL script into database.
6List Tables.tablesOutputs all database table names.
7Show Table Schema.schema usersDisplays table creation DDL syntax.
8Enable Header Display.headers onDisplays column headers in CLI.
9Set CSV Output Mode.mode csvChanges output format to comma-separated.
10Export CSV File.import --csv data.csv usersBulk imports CSV file into table.
11Analyze Database StatsANALYZE;Gathers query planner statistics.
12Vacuum DefragmentVACUUM;Rebuilds database file to reclaim space.
13Check IntegrityPRAGMA integrity_check;Scans database file for corruption.
14Set Journal Mode WALPRAGMA journal_mode = WAL;Enables Write-Ahead Logging concurrency.
15Check Journal ModePRAGMA journal_mode;Inspects active journal mode.
16Set Synchronous ModePRAGMA synchronous = NORMAL;Configures disk write synchronization.
17Check Page SizePRAGMA page_size;Inspects database page size in bytes.
18Set Cache SizePRAGMA cache_size = -64000;Allocates 64MB memory page cache.
19Explain Query ExecutionEXPLAIN QUERY PLAN SELECT * FROM users WHERE id = 1;Evaluates index search strategy.
20Attach External DatabaseATTACH DATABASE 'archive.db' AS archive;Mounts secondary database file.
21Detach External DatabaseDETACH DATABASE archive;Unmounts external database file.
22Create IndexCREATE INDEX idx_user_email ON users(email);Provisions table index.
23Drop IndexDROP INDEX idx_user_email;Removes table index.
24Enable Foreign KeysPRAGMA foreign_keys = ON;Enforces relational foreign key constraints.
25Check Foreign Key StatusPRAGMA foreign_keys;Inspects foreign key enforcement state.
26List Indexes on TablePRAGMA index_list('users');Outputs table index definitions.
27List Table ColumnsPRAGMA table_info('users');Inspects table column data types.
28Check Database SizeSELECT page_count * page_size FROM pragma_page_count(), pragma_page_size();Calculates database file disk size.
29Set Busy TimeoutPRAGMA busy_timeout = 5000;Waits 5 seconds for locked database locks.
30Exit SQLite CLI.exitCloses interactive shell.
31Quick Integrity CheckPRAGMA quick_check;Fast integrity scan omitting index validation.
32Check Foreign Key ViolationsPRAGMA foreign_key_check;Scans database tables for broken FK references.
33Set Auto VacuumPRAGMA auto_vacuum = FULL;Enforces automatic database page reclamation.
34Check Auto Vacuum ModePRAGMA auto_vacuum;Inspects active auto-vacuum setting.
35Incremental VacuumPRAGMA incremental_vacuum(100);Reclaims up to 100 free pages from database.
36Set Mmap SizePRAGMA mmap_size = 268435456;Allocates 256MB memory-mapped I/O limit.
37Check Mmap SizePRAGMA mmap_size;Inspects memory-mapped I/O limit.
38Set Locking ModePRAGMA locking_mode = EXCLUSIVE;Enforces exclusive file locking for performance.
39Reset Locking ModePRAGMA locking_mode = NORMAL;Restores shared file locking.
40Set Temp Store in MemoryPRAGMA temp_store = MEMORY;Forces temporary tables and indexes into RAM.
41Check Temp Store SettingPRAGMA temp_store;Inspects location setting for temp objects.
42Set WAL CheckpointPRAGMA wal_checkpoint(TRUNCATE);Flushes WAL file and truncates it to 0 bytes.
43Check User VersionPRAGMA user_version;Inspects application-defined schema version integer.
44Set User VersionPRAGMA user_version = 5;Sets application schema version integer.
45Check Application IDPRAGMA application_id;Inspects application identifier integer.
46Set Application IDPRAGMA application_id = 123456;Sets application identifier integer.
47Check Compile OptionsPRAGMA compile_options;Lists compiler flags used to build SQLite binary.
48Check Database EncodingPRAGMA encoding;Outputs database text encoding (UTF-8, UTF-16).
49Check Hard Heap LimitPRAGMA soft_heap_limit = 104857600;Sets 100MB soft memory heap limit.
50Check Freed List CountPRAGMA freelist_count;Displays number of unused free pages in DB file.
51Check Max Page CountPRAGMA max_page_count;Inspects maximum allowed pages in database file.
52Set Max Page CountPRAGMA max_page_count = 1000000;Limits maximum database file size.
53Check Schema VersionPRAGMA schema_version;Inspects internal schema modification counter.
54Check Secure DeletePRAGMA secure_delete = ON;Overwrites deleted content with zeroes.
55Set Threads CountPRAGMA threads = 4;Enables multi-threaded sorting and helper tasks.
56Set Column Output Mode.mode columnFormats CLI output into aligned text columns.
57Set JSON Output Mode.mode jsonFormats CLI query results as JSON array.
58Set HTML Output Mode.mode htmlFormats CLI query results as HTML table tags.
59Set List Output Mode.mode listFormats CLI query results as pipe-separated list.
60Set Markdown Output Mode.mode markdownFormats CLI query results as Markdown table.
61Set Output File.output output.txtRedirects CLI query results to text file.
62Restore Output Screen.output stdoutRestores CLI query results to screen stdout.
63Show Active Databases.databasesLists primary, temp, and attached database files.
64Show CLI Prompt Setting.prompt 'sqlite> ' ' ...> 'Customizes interactive shell prompt strings.
65Show CLI Timeout.timeout 10000Sets CLI lock wait timeout to 10 seconds.
66Check Table CountSELECT COUNT(*) FROM sqlite_master WHERE type='table';Calculates total table count in schema.
67List Table Indexes via MasterSELECT name FROM sqlite_master WHERE type='index' AND tbl_name='users';Queries system catalog for table indexes.
68List Triggers via MasterSELECT name FROM sqlite_master WHERE type='trigger';Lists all triggers in database.
69List Views via MasterSELECT name FROM sqlite_master WHERE type='view';Lists all views in database.
70Create ViewCREATE VIEW v_active_users AS SELECT * FROM users WHERE active = 1;Provisions virtual table view.
71Drop ViewDROP VIEW IF EXISTS v_active_users;Deletes virtual table view.
72Create TriggerCREATE TRIGGER trg_user_bu BEFORE UPDATE ON users BEGIN SELECT RAISE(FAIL, 'Updates forbidden'); END;Provisions pre-update validation trigger.
73Drop TriggerDROP TRIGGER IF EXISTS trg_user_bu;Deletes trigger routine.
74Create Unique IndexCREATE UNIQUE INDEX idx_uniq_email ON users(email);Enforces unique constraint via index.
75Create Partial IndexCREATE INDEX idx_active ON users(created_at) WHERE active = 1;Provisions index on subset of rows.
76Create Virtual Table FTS5CREATE VIRTUAL TABLE docs_fts USING fts5(title, body);Provisions full-text search 5 table.
77Insert FTS5 RecordINSERT INTO docs_fts (title, body) VALUES ('Doc 1', 'Database architecture guide');Writes text record into FTS5 index.
78Query FTS5 Fulltext SearchSELECT * FROM docs_fts WHERE docs_fts MATCH 'architecture';Executes high-speed full-text search.
79Optimize FTS5 IndexINSERT INTO docs_fts(docs_fts) VALUES('optimize');Compacts FTS5 index storage structures.
80Create Virtual Table RTreeCREATE VIRTUAL TABLE geo_index USING rtree(id, minX, maxX, minY, maxY);Provisions spatial R-Tree index table.
81Insert Spatial Box RTreeINSERT INTO geo_index VALUES (1, -73.9, -73.8, 40.7, 40.8);Writes bounding box into RTree index.
82Query Spatial Bounding BoxSELECT id FROM geo_index WHERE minX >= -74.0 AND maxX <= -73.0;Executes spatial bounding box search.
83Reindex All TablesREINDEX;Rebuilds all indexes across all attached databases.
84Reindex Specific TableREINDEX users;Rebuilds all indexes for specific table.
85Reindex Specific IndexREINDEX idx_user_email;Rebuilds a single index file.
86Check DB Page CountPRAGMA page_count;Inspects total pages allocated in DB file.
87Check DB Free ListPRAGMA freelist_count;Inspects free pages available for reuse.
88Check Table Column DefaultsSELECT dflt_value FROM pragma_table_info('users') WHERE name='status';Inspects default column value.
89Check Table Primary KeySELECT name FROM pragma_table_info('users') WHERE pk > 0;Identifies primary key columns.
90Check Foreign Key ListPRAGMA foreign_key_list('orders');Inspects target tables and referenced columns.
91Check Index Info DetailsPRAGMA index_info('idx_user_email');Inspects indexed columns and sort order.
92Check Index XInfo DetailsPRAGMA index_xinfo('idx_user_email');Inspects extended index properties.
93Savepoint TransactionSAVEPOINT sp1;Provisions transaction savepoint.
94Release SavepointRELEASE SAVEPOINT sp1;Commits transaction savepoint.
95Rollback to SavepointROLLBACK TO SAVEPOINT sp1;Reverts transaction state to savepoint.
96Begin Immediate TransactionBEGIN IMMEDIATE TRANSACTION;Acquires reserved lock before writes.
97Begin Exclusive TransactionBEGIN EXCLUSIVE TRANSACTION;Acquires exclusive write lock on DB file.
98Commit TransactionCOMMIT;Finalizes active transaction.
99Rollback TransactionROLLBACK;Aborts active transaction.
100Exit Shell Quietly.quitExits command-line tool.
101Quick Integrity CheckPRAGMA quick_check;Fast integrity scan omitting index validation.
102Check Foreign Key ViolationsPRAGMA foreign_key_check;Scans database tables for broken FK references.
103Set Auto VacuumPRAGMA auto_vacuum = FULL;Enforces automatic database page reclamation.
104Incremental VacuumPRAGMA incremental_vacuum(100);Reclaims up to 100 free pages from database.
105Set Mmap SizePRAGMA mmap_size = 268435456;Allocates 256MB memory-mapped I/O limit.
106Set Locking Mode ExclusivePRAGMA locking_mode = EXCLUSIVE;Enforces exclusive file locking for performance.
107Set Temp Store MemoryPRAGMA temp_store = MEMORY;Forces temporary tables and indexes into RAM.
108WAL Checkpoint TruncatePRAGMA wal_checkpoint(TRUNCATE);Flushes WAL file and truncates it to 0 bytes.
109Check User VersionPRAGMA user_version;Inspects application-defined schema version integer.
110Set User VersionPRAGMA user_version = 5;Sets application schema version integer.
111Check Database EncodingPRAGMA encoding;Outputs database text encoding (UTF-8, UTF-16).
112Set Threads CountPRAGMA threads = 4;Enables multi-threaded sorting and helper tasks.
113Set Output Mode JSON.mode jsonFormats CLI query results as JSON array.
114Set Output Mode Column.mode columnFormats CLI query results as aligned columns.
115Show Active Databases.databasesLists primary, temp, and attached database files.
116Check Table CountSELECT COUNT(*) FROM sqlite_master WHERE type='table';Calculates total table count in schema.
117Create ViewCREATE VIEW v_active_users AS SELECT * FROM users WHERE active = 1;Provisions virtual table view.
118Drop ViewDROP VIEW IF EXISTS v_active_users;Deletes virtual table view.
119Create FTS5 TableCREATE VIRTUAL TABLE docs_fts USING fts5(title, body);Provisions full-text search 5 table.
120Insert FTS5 RecordINSERT INTO docs_fts (title, body) VALUES ('Doc 1', 'Database architecture guide');Writes text record into FTS5 index.
121Query FTS5 SearchSELECT * FROM docs_fts WHERE docs_fts MATCH 'architecture';Executes high-speed full-text search.
122Optimize FTS5 IndexINSERT INTO docs_fts(docs_fts) VALUES('optimize');Compacts FTS5 index storage structures.
123Create RTree Spatial TableCREATE VIRTUAL TABLE geo_index USING rtree(id, minX, maxX, minY, maxY);Provisions spatial R-Tree index table.
124Reindex All TablesREINDEX;Rebuilds all indexes across all attached databases.
125Check DB Page CountPRAGMA page_count;Inspects total pages allocated in DB file.
126Check DB Free ListPRAGMA freelist_count;Inspects free pages available for reuse.
127Begin Immediate TransactionBEGIN IMMEDIATE TRANSACTION;Acquires reserved lock before writes.
128Begin Exclusive TransactionBEGIN EXCLUSIVE TRANSACTION;Acquires exclusive write lock on DB file.
129Commit TransactionCOMMIT;Finalizes active transaction.
130Exit Shell Quietly.quitExits command-line tool.
131Check Database Page CountPRAGMA page_count;Inspects total allocated pages in DB file.
132Check Database Free Page CountPRAGMA freelist_count;Inspects unallocated free pages available for reuse.
133Check Database Page SizePRAGMA page_size;Inspects page size in bytes (default 4096).
134Set Database Page SizePRAGMA page_size = 8192;Configures 8KB page size for new database files.
135Check Database Cache SizePRAGMA cache_size;Inspects active page cache memory limit.
136Set Database Cache Size KBPRAGMA cache_size = -128000;Allocates 128MB RAM for page cache.
137Check Database Max Page CountPRAGMA max_page_count;Inspects maximum allowed pages in database file.
138Set Database Max Page CountPRAGMA max_page_count = 2000000;Caps database file size to ~8GB.
139Check Database Busy TimeoutPRAGMA busy_timeout;Inspects lock wait timeout in milliseconds.
140Set Database Busy TimeoutPRAGMA busy_timeout = 10000;Sets 10-second lock wait timeout before throwing SQLITE_BUSY.
141Check Database Synchronous SettingPRAGMA synchronous;Inspects disk write sync mode (0=OFF, 1=NORMAL, 2=FULL, 3=EXTRA).
142Set Database Synchronous OffPRAGMA synchronous = OFF;Disables disk sync for maximum write speed in temp DBs.
143Set Database Synchronous FullPRAGMA synchronous = FULL;Enforces full disk sync for maximum crash durability.
144Check Database Journal ModePRAGMA journal_mode;Inspects active journal mode (DELETE, TRUNCATE, PERSIST, MEMORY, WAL, OFF).
145Set Database Journal Mode DeletePRAGMA journal_mode = DELETE;Restores traditional rollback journal mode.
146Set Database Journal Mode MemoryPRAGMA journal_mode = MEMORY;Stores rollback journal in RAM.
147Check Database WAL Auto CheckpointPRAGMA wal_autocheckpoint;Inspects WAL auto-checkpoint page threshold (default 1000).
148Set Database WAL Auto CheckpointPRAGMA wal_autocheckpoint = 5000;Sets auto-checkpoint trigger to 5000 pages.
149Check Database Lock StatusPRAGMA lock_status;Inspects active read/write file locks held.
150Check Database Temp StorePRAGMA temp_store;Inspects storage location for temp tables (0=DEFAULT, 1=FILE, 2=MEMORY).
151Set Database Temp Store FilePRAGMA temp_store = FILE;Forces temp tables into disk files.
152Check Database Foreign KeysPRAGMA foreign_keys;Inspects foreign key enforcement setting.
153Disable Database Foreign KeysPRAGMA foreign_keys = OFF;Disables foreign key constraint checks during bulk loads.
154Check Database Recursive TriggersPRAGMA recursive_triggers;Inspects recursive trigger execution setting.
155Enable Database Recursive TriggersPRAGMA recursive_triggers = ON;Allows triggers to fire recursively.
156Check Database Secure DeletePRAGMA secure_delete;Inspects whether deleted pages are zero-overwritten.
157Disable Database Secure DeletePRAGMA secure_delete = OFF;Disables zero-overwrite on deleted pages for speed.
158Check Database Mmap SizePRAGMA mmap_size;Inspects memory-mapped I/O byte limit.
159Set Database Mmap Size 1GBPRAGMA mmap_size = 1073741824;Allocates 1GB memory-mapped I/O limit.
160Check Database ThreadsPRAGMA threads;Inspects helper thread pool limit.
161Check Database User VersionPRAGMA user_version;Inspects user schema version integer.
162Check Database Application IDPRAGMA application_id;Inspects application ID integer.
163Check Database Schema VersionPRAGMA schema_version;Inspects internal schema modification sequence.
164Check Database EncodingPRAGMA encoding;Outputs text encoding (UTF-8, UTF-16le, UTF-16be).
165Check Database Compile OptionsPRAGMA compile_options;Outputs list of C compiler flags used to build SQLite.
166Check Database Soft Heap LimitPRAGMA soft_heap_limit;Inspects soft memory heap limit in bytes.
167Set Database Soft Heap LimitPRAGMA soft_heap_limit = 2147483648;Sets 2GB soft memory heap limit.
168Check Table Column InfoPRAGMA table_info('users');Outputs column names, types, nullability, and primary key status.
169Check Table Extended Column InfoPRAGMA table_xinfo('users');Outputs column details including generated columns.
170Check Table Index ListPRAGMA index_list('users');Lists all secondary and primary indexes on table.
171Check Index Column DetailsPRAGMA index_info('idx_users_email');Lists columns included in specific index.
172Check Index Extended DetailsPRAGMA index_xinfo('idx_users_email');Lists key and non-key columns in index.
173Check Foreign Key ReferencesPRAGMA foreign_key_list('orders');Lists foreign key target tables and columns.
174Check Database Integrity FullPRAGMA integrity_check;Scans entire database file for structural corruption.
175Check Database Integrity QuickPRAGMA quick_check;Scans database file omitting index leaf validation.
176Check Database Foreign Key ViolationsPRAGMA foreign_key_check;Scans entire database for broken foreign key references.
177Check Table Foreign Key ViolationsPRAGMA foreign_key_check('orders');Scans specific table for broken foreign key references.
178Checkpoint WAL PassivePRAGMA wal_checkpoint(PASSIVE);Checkpoints WAL pages without blocking active readers/writers.
179Checkpoint WAL FullPRAGMA wal_checkpoint(FULL);Checkpoints all WAL pages waiting for readers to finish.
180Checkpoint WAL RestartPRAGMA wal_checkpoint(RESTART);Flushes WAL and blocks new readers to restart WAL at byte 0.
181Checkpoint WAL TruncatePRAGMA wal_checkpoint(TRUNCATE);Flushes WAL, blocks readers, and truncates file to 0 bytes.
182Set CLI Mode Table.mode tableFormats CLI query output as ASCII box table.
183Set CLI Mode Box.mode boxFormats CLI query output as Unicode box table.
184Set CLI Mode CSV.mode csvFormats CLI query output as comma-separated values.
185Set CLI Mode JSON.mode jsonFormats CLI query output as JSON array.
186Set CLI Mode Line.mode lineFormats CLI query output as key-value lines.
187Set CLI Mode List.mode listFormats CLI query output as pipe-separated list.
188Set CLI Mode Markdown.mode markdownFormats CLI query output as Markdown table.
189Set CLI Mode HTML.mode htmlFormats CLI query output as HTML table tags.
190Set CLI Output File.output results.txtRedirects query results output to text file.
191Restore CLI Output Screen.output stdoutRestores query results output to terminal screen.
192Show CLI Tables List.tablesLists all tables in connected databases.
193Show CLI Schema DDL.schemaOutputs full DDL schema for all tables and indexes.
194Show Table DDL.schema usersOutputs DDL syntax for specific table.
195Show Active Database Files.databasesLists primary, temp, and attached database files.
196Show CLI Headers On.headers onEnables column headers in query output.
197Show CLI Headers Off.headers offDisables column headers in query output.
198Check SQLite Version SQLSELECT sqlite_version();Outputs running SQLite engine release version string.
199Check SQLite Source ID SQLSELECT sqlite_source_id();Outputs exact git commit hash used to build SQLite.
200Check Connection Ping SQLSELECT 1;Simple connection ping test returning 1.

CockroachDB OLTP

Technical Architecture & Overview

CockroachDB is a cloud-native, distributed SQL database designed for global scalability and strong consistency. It uses a transactional key-value store layered under a distributed SQL query engine, leveraging the Raft consensus algorithm for Multi-Raft data replication and Range splitting.

Primary Use Cases: Multi-region cloud applications, global SaaS platforms, distributed financial ledgers, and scale-out transactional services requiring serializable isolation.

Core Components: SQL Layer, Distributed KV Layer, Raft Consensus Protocol, Storage Layer (Pebble engine), Range Allocator, and System Jobs Manager.

Exhaustive 200-command administrative, diagnostic, and operational matrix for CockroachDB.

#Operation / ActionCLI / SQL SyntaxDescription
1Start Node Servercockroach start --insecure --store=node1 --listen-addr=localhost:26257Boots distributed database node.
2Initialize Clustercockroach init --insecure --host=localhost:26257Bootstraps new distributed cluster.
3Interactive SQL Shellcockroach sql --insecure --host=localhost:26257Opens distributed SQL monitor.
4Export Cluster BackupBACKUP INTO 's3://my-bucket/backup' AS OF SYSTEM TIME '-1m';Takes distributed cluster snapshot.
5Restore Cluster BackupRESTORE FROM 's3://my-bucket/backup';Restores distributed database snapshot.
6Check Node StatusSHOW CONSTRAINTS; SELECT * FROM crdb_internal.gossip_nodes;Inspects active cluster nodes.
7Check Range ReplicationSHOW RANGES FROM TABLE users;Inspects distributed data raft ranges.
8Create DatabaseCREATE DATABASE enterprise_db;Provisions new distributed database.
9Drop DatabaseDROP DATABASE enterprise_db;Deletes distributed database.
10Create UserCREATE USER app_user WITH PASSWORD 'SecurePass!';Provisions cluster user login.
11Grant PrivilegesGRANT ALL ON DATABASE enterprise_db TO app_user;Assigns distributed data access.
12Explain Query ExecutionEXPLAIN ANALYZE SELECT * FROM users WHERE id = 5;Evaluates distributed execution plan.
13Check Table StatsSHOW STATISTICS FOR TABLE users;Inspects cost-based optimizer statistics.
14Create IndexCREATE INDEX idx_user_email ON users(email);Provisions distributed table index.
15Drop IndexDROP INDEX users@idx_user_email;Removes distributed index.
16Set Zone ConfigurationALTER RANGE default CONFIGURE ZONE USING num_replicas = 5;Configures data replication zones.
17View Zone ConfigsSHOW ZONE CONFIGURATION FOR DATABASE enterprise_db;Inspects replication zone rules.
18Check Cluster MetricsSELECT * FROM crdb_internal.node_metrics WHERE metric LIKE 'sql.query%';Pulls cluster telemetry metrics.
19Decommission Nodecockroach node decommission 2 --insecureSafely removes node from cluster.
20Inspect Node LivenessSELECT * FROM crdb_internal.liveness;Checks node heartbeat status.
21Pause Job ExecutionCANCEL JOB 8492038420;Stops background schema change job.
22List Running JobsSHOW JOBS;Inspects asynchronous cluster jobs.
23Set Session SettingSET application_name = 'EnterpriseBatch';Configures connection metadata.
24Create SchemaCREATE SCHEMA staging;Provisions logical database namespace.
25Set Default TTLALTER TABLE logs SET (ttl_expire_after = '90 days');Enables automatic data expiration.
26Inspect KV Storage SizeSELECT sum(range_size) FROM crdb_internal.ranges;Calculates physical cluster storage size.
27Verify Cluster VersionSHOW CLUSTER SETTING version;Inspects running software version.
28Set Cluster SettingSET CLUSTER SETTING kv.rangefeed.enabled = true;Modifies global cluster parameter.
29Trace Query ExecutionEXPLAIN DISTSQL SELECT * FROM users;Inspects distributed query flow.
30Quit SQL Shell\qCloses interactive shell session.
31Check Active SessionsSHOW SESSIONS;Inspects connected client session threads.
32Check Active QueriesSHOW STATEMENTS;Views currently executing SQL queries.
33Cancel SessionCANCEL SESSION '16834abc';Terminates active client connection session.
34Cancel StatementCANCEL STATEMENT '16834abc';Terminates specific running SQL query.
35Pause JobPAUSE JOB 8492038420;Temporarily halts execution of background job.
36Resume JobRESUME JOB 8492038420;Resumes execution of paused background job.
37Show Job DetailsSHOW JOB 8492038420;Inspects progress and error details of background job.
38Create Inverted IndexCREATE INVERTED INDEX idx_payload ON logs(payload);Provisions GIN-like index for JSONB columns.
39Create Stored ProcedureCREATE PROCEDURE clean_data() AS $$ BEGIN DELETE FROM logs WHERE created_at < NOW() - INTERVAL '30 days'; END; $$ LANGUAGE PLpgSQL;Provisions stored procedure routine.
40Call Stored ProcedureCALL clean_data();Executes stored procedure routine.
41Drop Stored ProcedureDROP PROCEDURE clean_data;Deletes stored procedure routine.
42Create User Defined FunctionCREATE FUNCTION add_tax(p DECIMAL) RETURNS DECIMAL AS $$ SELECT p * 1.08; $$ LANGUAGE SQL;Provisions scalar user function.
43Drop User Defined FunctionDROP FUNCTION add_tax;Deletes user defined function.
44Create Changefeed (CDC)CREATE CHANGEFEED FOR TABLE users INTO 'kafka://localhost:9092';Streams CDC row mutations to Kafka.
45Create Changefeed with FilterCREATE CHANGEFEED FOR TABLE users WHERE email LIKE '%@org.com' INTO 'kafka://localhost:9092';Streams filtered CDC events.
46Show ChangefeedsSHOW CHANGEFEEDS;Lists active CDC changefeed pipelines.
47Cancel ChangefeedCANCEL JOB (SELECT job_id FROM [SHOW CHANGEFEEDS] WHERE topic = 'users');Stops running CDC changefeed.
48Create Regional TableALTER TABLE users SET LOCALITY REGIONAL BY TABLE IN "us-east1";Binds table storage to specific cloud region.
49Create Global TableALTER TABLE reference_data SET LOCALITY GLOBAL;Optimizes table for fast low-latency reads worldwide.
50Create Regional Row TableALTER TABLE users SET LOCALITY REGIONAL BY ROW AS region;Binds individual rows to cloud region.
51Show Table LocalitySHOW LOCALITY FROM TABLE users;Inspects multi-region locality configuration.
52Add Region to DatabaseALTER DATABASE enterprise_db ADD REGION "us-west1";Expands database multi-region footprint.
53Drop Region from DatabaseALTER DATABASE enterprise_db DROP REGION "us-west1";Removes cloud region from database.
54Set Primary RegionALTER DATABASE enterprise_db SET PRIMARY REGION "us-east1";Designates primary region for database metadata.
55Show Database RegionsSHOW REGIONS FROM DATABASE enterprise_db;Lists cloud regions assigned to database.
56Check Range LeasesSELECT * FROM crdb_internal.range_leases;Inspects active Raft leaseholders per range.
57Check Range ReplicasSELECT * FROM crdb_internal.kv_node_status;Inspects KV node status and Raft replica count.
58Check Range SplittingALTER TABLE users CONFIGURE ZONE USING range_max_bytes = 67108864;Configures 64MB max Range split size.
59Split Range ManuallyALTER TABLE users SPLIT AT VALUES (100000);Manually forces range split at key boundary.
60Unsplit Range ManuallyALTER TABLE users UNSPLIT AT VALUES (100000);Removes manual range split boundary.
61Check Cluster SettingsSHOW ALL CLUSTER SETTINGS;Lists all configurable cluster parameters.
62Reset Cluster SettingRESET CLUSTER SETTING kv.rangefeed.enabled;Restores cluster setting to default value.
63Check Table LocksSELECT * FROM crdb_internal.locks;Inspects lock contention across KV keys.
64Check Contention EventsSELECT * FROM crdb_internal.transaction_contention_events;Analyzes transaction retry bottlenecks.
65Set Transaction IsolationSET TRANSACTION ISOLATION LEVEL SERIALIZABLE;Enforces strict serializable isolation.
66Set Transaction PrioritySET TRANSACTION PRIORITY HIGH;Increases transaction priority to reduce retries.
67Show Table Schema DDLSHOW CREATE TABLE users;Outputs exact DDL syntax used to build table.
68Show Database Schema DDLSHOW CREATE DATABASE enterprise_db;Outputs exact DDL syntax used to build DB.
69Check Indexes on TableSHOW INDEXES FROM users;Lists all primary and secondary index keys.
70Create Partial IndexCREATE INDEX idx_active_users ON users(created_at) WHERE active = true;Provisions filtered index for active rows.
71Create Covering IndexCREATE INDEX idx_email_name ON users(email) STORING (name);Includes non-key columns in index storage.
72Create Hash Partitioned IndexCREATE INDEX idx_id_hash ON users(id) USING HASH;Distributes index traffic across ranges.
73Check Disk Usage per TableSELECT table_name, pg_size_pretty(total_bytes) FROM crdb_internal.table_sizes;Calculates disk usage per table.
74Check Disk Usage per NodeSELECT node_id, pg_size_pretty(available_bytes) FROM crdb_internal.kv_node_status;Calculates disk space per node.
75Create RoleCREATE ROLE read_only_role;Provisions access role container.
76Grant Role to UserGRANT read_only_role TO app_user;Assigns role permissions to user.
77Revoke Role from UserREVOKE read_only_role FROM app_user;Revokes role permissions from user.
78Drop RoleDROP ROLE read_only_role;Deletes role container.
79Grant Schema UsageGRANT USAGE ON SCHEMA staging TO app_user;Grants namespace access rights.
80Revoke Schema UsageREVOKE USAGE ON SCHEMA staging FROM app_user;Revokes namespace access rights.
81Show Grants for UserSHOW GRANTS FOR app_user;Lists all privileges assigned to user.
82Set Password ExpiryALTER USER app_user WITH PASSWORD 'Pass123!' VALID UNTIL '2027-01-01';Sets login expiration date.
83Check System Jobs HistorySELECT * FROM crdb_internal.jobs WHERE job_type = 'BACKUP';Inspects backup job history log.
84Check Node Hardware SpecsSELECT * FROM crdb_internal.node_hardware;Inspects CPU cores and RAM per node.
85Check Hot RangesSELECT * FROM crdb_internal.hot_ranges;Identifies ranges receiving high QPS traffic.
86Check Table Column TypesSHOW COLUMNS FROM users;Inspects table column definitions.
87Rename TableALTER TABLE users RENAME TO app_users;Renames table relation in database.
88Rename ColumnALTER TABLE users RENAME COLUMN email TO user_email;Renames table column.
89Add Column DefaultALTER TABLE users ALTER COLUMN status SET DEFAULT 'active';Sets default value for column.
90Drop Column DefaultALTER TABLE users ALTER COLUMN status DROP DEFAULT;Removes default value from column.
91Set Column TypeALTER TABLE users ALTER COLUMN age SET DATA TYPE INT8;Changes column data type definition.
92Create SavepointSAVEPOINT cockroach_restart;Provisions savepoint for application retries.
93Release SavepointRELEASE SAVEPOINT cockroach_restart;Finalizes transaction retry savepoint.
94Rollback SavepointROLLBACK TO SAVEPOINT cockroach_restart;Retries transaction after conflict.
95Check Cluster LicenseSHOW CLUSTER SETTING enterprise.license;Inspects enterprise license key status.
96Set Enterprise LicenseSET CLUSTER SETTING enterprise.license = 'ccl_key_xyz';Applies enterprise feature key.
97Check Cluster CPU TelemetrySELECT * FROM crdb_internal.node_metrics WHERE metric LIKE 'sys.cpu%';Inspects CPU load across cluster.
98Check Cluster Memory TelemetrySELECT * FROM crdb_internal.node_metrics WHERE metric LIKE 'sys.rss%';Inspects RAM usage across cluster.
99Flush Table StatsANALYZE users;Triggers on-demand optimizer statistics collection.
100Node Health Check HTTPcurl http://localhost:8080/healthQueries HTTP diagnostic endpoint for node status.
101Check Active SessionsSHOW SESSIONS;Inspects connected client session threads.
102Check Active StatementsSHOW STATEMENTS;Views currently executing SQL queries.
103Cancel StatementCANCEL STATEMENT '16834abc';Terminates specific running SQL query.
104Pause Background JobPAUSE JOB 8492038420;Temporarily halts execution of background job.
105Resume Background JobRESUME JOB 8492038420;Resumes execution of paused background job.
106Show Job DetailsSHOW JOB 8492038420;Inspects progress and error details of background job.
107Create Inverted IndexCREATE INVERTED INDEX idx_payload ON logs(payload);Provisions GIN-like index for JSONB columns.
108Create Changefeed (CDC)CREATE CHANGEFEED FOR TABLE users INTO 'kafka://localhost:9092';Streams CDC row mutations to Kafka.
109Show Active ChangefeedsSHOW CHANGEFEEDS;Lists active CDC changefeed pipelines.
110Create Regional TableALTER TABLE users SET LOCALITY REGIONAL BY TABLE IN "us-east1";Binds table storage to specific cloud region.
111Create Global TableALTER TABLE reference_data SET LOCALITY GLOBAL;Optimizes table for fast low-latency reads worldwide.
112Create Regional Row TableALTER TABLE users SET LOCALITY REGIONAL BY ROW AS region;Binds individual rows to cloud region.
113Show Table LocalitySHOW LOCALITY FROM TABLE users;Inspects multi-region locality configuration.
114Add Region to DBALTER DATABASE enterprise_db ADD REGION "us-west1";Expands database multi-region footprint.
115Drop Region from DBALTER DATABASE enterprise_db DROP REGION "us-west1";Removes cloud region from database.
116Set Primary RegionALTER DATABASE enterprise_db SET PRIMARY REGION "us-east1";Designates primary region for database metadata.
117Show Database RegionsSHOW REGIONS FROM DATABASE enterprise_db;Lists cloud regions assigned to database.
118Check Range LeasesSELECT * FROM crdb_internal.range_leases;Inspects active Raft leaseholders per range.
119Check Range ReplicasSELECT * FROM crdb_internal.kv_node_status;Inspects KV node status and Raft replica count.
120Split Range ManuallyALTER TABLE users SPLIT AT VALUES (100000);Manually forces range split at key boundary.
121Unsplit Range ManuallyALTER TABLE users UNSPLIT AT VALUES (100000);Removes manual range split boundary.
122Check Cluster SettingsSHOW ALL CLUSTER SETTINGS;Lists all configurable cluster parameters.
123Check Transaction ContentionSELECT * FROM crdb_internal.transaction_contention_events;Analyzes transaction retry bottlenecks.
124Set Isolation LevelSET TRANSACTION ISOLATION LEVEL SERIALIZABLE;Enforces strict serializable isolation.
125Show Create Table DDLSHOW CREATE TABLE users;Outputs exact DDL syntax used to build table.
126Create Covering IndexCREATE INDEX idx_email_name ON users(email) STORING (name);Includes non-key columns in index storage.
127Check Disk Usage per TableSELECT table_name, pg_size_pretty(total_bytes) FROM crdb_internal.table_sizes;Calculates disk usage per table.
128Check Disk Usage per NodeSELECT node_id, pg_size_pretty(available_bytes) FROM crdb_internal.kv_node_status;Calculates disk space per node.
129Check Hot RangesSELECT * FROM crdb_internal.hot_ranges;Identifies ranges receiving high QPS traffic.
130Check Node Health HTTPcurl http://localhost:8080/healthQueries HTTP diagnostic endpoint for node status.
131Check Cluster IDSELECT * FROM crdb_internal.cluster_id;Outputs unique cluster GUID.
132Check Cluster Nodes ListSELECT node_id, address, build_tag FROM crdb_internal.gossip_nodes;Lists all active nodes in cluster gossip.
133Check Node Network LivenessSELECT node_id, is_live, updated_at FROM crdb_internal.liveness;Monitors node heartbeat timestamps.
134Check Range Count TotalSELECT count(*) FROM crdb_internal.ranges;Calculates total Raft range count in cluster.
135Check Range Size MetricsSELECT min(range_size), avg(range_size), max(range_size) FROM crdb_internal.ranges;Calculates min, avg, and max Raft range byte sizes.
136Check Range Under Replicated CountSELECT count(*) FROM crdb_internal.ranges WHERE array_length(replicas, 1) < 3;Identifies under-replicated ranges.
137Check Range Unavailable CountSELECT count(*) FROM crdb_internal.ranges WHERE array_length(replicas, 1) = 0;Identifies offline unavailable ranges.
138Check Table Partition RangesSELECT * FROM crdb_internal.ranges WHERE table_name = 'users';Inspects Raft ranges belonging to specific table.
139Check Raft Leader DistributionSELECT lease_holder, count(*) FROM crdb_internal.ranges GROUP BY lease_holder;Calculates Raft leaseholder distribution across nodes.
140Check Table Column MetadataSELECT column_name, data_type, is_nullable FROM information_schema.columns WHERE table_name = 'users';Inspects column definitions.
141Check Table Primary Key MetadataSELECT column_name FROM information_schema.key_column_usage WHERE table_name = 'users';Identifies primary key columns.
142Check Table Foreign Key MetadataSELECT * FROM information_schema.referential_constraints;Lists foreign key relationships in database.
143Check Table Indexes MetadataSELECT index_name, column_name, direction FROM information_schema.statistics WHERE table_name = 'users';Lists indexed columns and sort directions.
144Check Table Constraints ListSELECT constraint_name, constraint_type FROM information_schema.table_constraints WHERE table_name = 'users';Lists unique, check, and foreign key constraints.
145Check System Jobs RunningSELECT job_id, description, status FROM crdb_internal.jobs WHERE status = 'running';Lists active background jobs.
146Check System Jobs FailedSELECT job_id, description, error FROM crdb_internal.jobs WHERE status = 'failed';Inspects failed background jobs and stack traces.
147Check System Jobs CompletedSELECT job_id, description, created, finished FROM crdb_internal.jobs WHERE status = 'succeeded' ORDER BY finished DESC LIMIT 10;Inspects recently completed jobs.
148Check Active SQL SessionsSELECT session_id, user_name, client_address, application_name FROM crdb_internal.node_sessions;Lists active client sessions across all nodes.
149Check Active SQL QueriesSELECT query_id, user_name, query, phase FROM crdb_internal.node_statement_statistics WHERE phase = 'executing';Lists active running SQL queries.
150Check Slow SQL QueriesSELECT query, service_lat, exec_lat FROM crdb_internal.node_statement_statistics ORDER BY exec_lat DESC LIMIT 10;Finds queries with highest execution latency.
151Check SQL Memory UsageSELECT node_id, value FROM crdb_internal.node_metrics WHERE metric = 'sql.mem.sql.current';Monitors RAM allocated to SQL query execution.
152Check KV Memory UsageSELECT node_id, value FROM crdb_internal.node_metrics WHERE metric = 'val.mem.pebble.current';Monitors RAM allocated to Pebble KV storage engine.
153Check Go Runtime Memory UsageSELECT node_id, value FROM crdb_internal.node_metrics WHERE metric = 'sys.rss';Monitors resident set size RAM per node.
154Check Go Runtime Garbage CollectionSELECT node_id, value FROM crdb_internal.node_metrics WHERE metric = 'sys.gc.pause.ns';Monitors Go GC pause latency.
155Check CPU User PercentSELECT node_id, value FROM crdb_internal.node_metrics WHERE metric = 'sys.cpu.user.percent';Monitors user-space CPU utilization percentage.
156Check CPU System PercentSELECT node_id, value FROM crdb_internal.node_metrics WHERE metric = 'sys.cpu.sys.percent';Monitors kernel-space CPU utilization percentage.
157Check Disk Free BytesSELECT node_id, value FROM crdb_internal.node_metrics WHERE metric = 'capacity.available';Monitors free storage disk space per node.
158Check Disk Total BytesSELECT node_id, value FROM crdb_internal.node_metrics WHERE metric = 'capacity';Monitors total storage disk space per node.
159Check Disk Used BytesSELECT node_id, value FROM crdb_internal.node_metrics WHERE metric = 'capacity.used';Monitors used storage disk space per node.
160Check Network Bytes SentSELECT node_id, value FROM crdb_internal.node_metrics WHERE metric = 'exec.net.bytes-sent';Monitors inter-node network egress bytes.
161Check Network Bytes ReceivedSELECT node_id, value FROM crdb_internal.node_metrics WHERE metric = 'exec.net.bytes-rcvd';Monitors inter-node network ingress bytes.
162Check Raft Proposals CountSELECT node_id, value FROM crdb_internal.node_metrics WHERE metric = 'raft.commands.applied';Monitors total Raft consensus state machine commits.
163Check Raft Command LatencySELECT node_id, value FROM crdb_internal.node_metrics WHERE metric = 'raft.process.logcommit.latency';Monitors Raft log write commit latency.
164Check Storage Engine Pebble CompactionSELECT node_id, value FROM crdb_internal.node_metrics WHERE metric = 'storage.compaction.count';Monitors background Pebble SSTable compactions.
165Check Storage Engine Pebble Write AmplificationSELECT node_id, value FROM crdb_internal.node_metrics WHERE metric = 'storage.write-amp';Monitors Pebble storage write amplification ratio.
166Check Session Application NameSHOW application_name;Outputs active connection application name.
167Check Session TimezoneSHOW TIME ZONE;Outputs active session time zone.
168Set Session TimezoneSET TIME ZONE 'UTC';Sets active session time zone to UTC.
169Check Session Transaction IsolationSHOW TRANSACTION ISOLATION LEVEL;Outputs active transaction isolation mode (SERIALIZABLE).
170Check Current User AccountSELECT current_user();Returns active execution user name.
171Check Current Database NameSELECT current_database();Returns active selected database name.
172Check Current Schema NameSELECT current_schema();Returns active selected schema name.
173Check Current TimestampSELECT clock_timestamp();Returns current wall-clock timestamp.
174Check CockroachDB Version SQLSELECT version();Outputs running CockroachDB build version and git commit hash.
175Check Connection Ping SQLSELECT 1;Simple connection ping test returning 1.
176Check Health Endpoint HTTPcurl http://localhost:8080/healthQueries HTTP REST endpoint for node health status.
177Check Health Readiness HTTPcurl http://localhost:8080/health?ready=1Queries HTTP REST endpoint for node readiness probe.
178Check Metrics Endpoint HTTPcurl http://localhost:8080/_status/varsPulls Prometheus metrics endpoint.
179Check CLI Node Statuscockroach node status --insecureOutputs status table for all nodes via CLI tool.
180Check CLI Cluster Versioncockroach versionOutputs client CLI binary version string.
181Check Table Column Default ValuesSELECT column_name, column_default FROM information_schema.columns WHERE table_name = 'users';Inspects column default values.
182Check Table Column Data TypesSELECT column_name, data_type FROM information_schema.columns WHERE table_name = 'users';Inspects column data types.
183Check Table Column NullabilitySELECT column_name, is_nullable FROM information_schema.columns WHERE table_name = 'users';Inspects column nullability.
184Check Table Primary Key ColumnsSELECT column_name FROM information_schema.key_column_usage WHERE table_name = 'users';Inspects primary key columns.
185Check Table Foreign Keys ListSELECT constraint_name FROM information_schema.table_constraints WHERE table_name = 'users' AND constraint_type = 'FOREIGN KEY';Lists foreign key constraints.
186Check Table Unique Constraints ListSELECT constraint_name FROM information_schema.table_constraints WHERE table_name = 'users' AND constraint_type = 'UNIQUE';Lists unique constraints.
187Check Table Check Constraints ListSELECT constraint_name FROM information_schema.table_constraints WHERE table_name = 'users' AND constraint_type = 'CHECK';Lists check constraints.
188Check Table Indexes ListSELECT index_name FROM information_schema.statistics WHERE table_name = 'users';Lists indexes on table.
189Check Table Storage Size BytesSELECT pg_relation_size('users');Calculates table storage size in bytes.
190Check Database Size BytesSELECT pg_database_size('enterprise_db');Calculates database storage size in bytes.
191Check Cluster Node CountSELECT count(*) FROM crdb_internal.gossip_nodes;Calculates total cluster node count.
192Check Cluster Live Node CountSELECT count(*) FROM crdb_internal.liveness WHERE is_live = true;Calculates total live cluster node count.
193Check Cluster Range CountSELECT count(*) FROM crdb_internal.ranges;Calculates total Raft range count.
194Check Cluster Range Replicas CountSELECT sum(array_length(replicas, 1)) FROM crdb_internal.ranges;Calculates total Raft range replicas.
195Check Cluster Job CountSELECT count(*) FROM crdb_internal.jobs;Calculates total background jobs count.
196Check Cluster Job Running CountSELECT count(*) FROM crdb_internal.jobs WHERE status = 'running';Calculates running jobs count.
197Check Cluster Job Failed CountSELECT count(*) FROM crdb_internal.jobs WHERE status = 'failed';Calculates failed jobs count.
198Check Cluster Job Succeeded CountSELECT count(*) FROM crdb_internal.jobs WHERE status = 'succeeded';Calculates succeeded jobs count.
199Check Session User NameSELECT current_user();Outputs current user name.
200Check Session Database NameSELECT current_database();Outputs current database name.

PingCAP TiDB OLTP

Technical Architecture & Overview

PingCAP TiDB is a distributed Hybrid Transactional and Analytical Processing (HTAP) database. It decouples stateless SQL compute nodes (TiDB) from distributed row storage nodes (TiKV, using Multi-Raft) and columnar analytical storage nodes (TiFlash, using Raft learner synchronization), governed by a Placement Driver (PD) cluster.

Primary Use Cases: Large-scale real-time HTAP workloads, financial core transactions, scale-out MySQL-compatible databases, and real-time reporting dashboards.

Core Components: TiDB Server (Stateless SQL), TiKV (Distributed Row Storage), TiFlash (Columnar Analytics), Placement Driver (PD Metadata & Timestamp Oracle), and TiUP CLI toolchain.

Exhaustive 200-command administrative, diagnostic, and operational matrix for PingCAP TiDB.

#Operation / ActionTiUP / SQL SyntaxDescription
1Deploy Cluster via TiUPtiup cluster deploy prod-cluster v8.0.0 topo.yaml --user rootDeploys distributed TiDB cluster.
2Start Clustertiup cluster start prod-clusterBoots TiDB, PD, and TiKV services.
3Stop Clustertiup cluster stop prod-clusterHalts distributed cluster services.
4Connect via MySQL Clientmysql -h 127.0.0.1 -P 4000 -u rootConnects to TiDB stateless SQL layer.
5Check Cluster Statustiup cluster display prod-clusterInspects cluster topology and nodes.
6Backup via BR Toolbr backup full --pd 127.0.0.1:2379 --storage "s3://backup-bucket/tidb"Executes distributed cluster backup.
7Restore via BR Toolbr restore full --pd 127.0.0.1:2379 --storage "s3://backup-bucket/tidb"Restores cluster backup data.
8Create DatabaseCREATE DATABASE enterprise_db;Provisions new distributed database.
9Drop DatabaseDROP DATABASE enterprise_db;Deletes distributed database.
10Create UserCREATE USER 'app_user'@'%' IDENTIFIED BY 'SecurePass!';Provisions cluster user.
11Grant PrivilegesGRANT ALL PRIVILEGES ON enterprise_db.* TO 'app_user'@'%';Assigns data access rights.
12Explain Query ExecutionEXPLAIN ANALYZE SELECT * FROM users WHERE id = 10;Evaluates distributed execution plan.
13Analyze Table StatsANALYZE TABLE users;Refreshes cost-based optimizer stats.
14Show Region DistributionSHOW TABLE users REGIONS;Inspects TiKV multi-raft region split.
15Check Slow QueriesSELECT * FROM information_schema.cluster_slow_query WHERE time > '2026-07-01';Pulls cluster performance bottlenecks.
16Check TiKV Storage SizeSELECT * FROM information_schema.tikv_store_status;Evaluates TiKV node disk utilization.
17Create IndexALTER TABLE users ADD INDEX idx_email (email);Provisions distributed index.
18Drop IndexALTER TABLE users DROP INDEX idx_email;Removes distributed index.
19Set TiFlash ReplicaALTER TABLE users SET TIFLASH REPLICA 2;Provisions columnar analytical replica.
20Check TiFlash StatusSELECT * FROM information_schema.tiflash_replica;Inspects columnar sync status.
21Check PD Leadercurl http://127.0.0.1:2379/pd/api/v1/leaderInspects Placement Driver leader node.
22Set Global ConfigurationSET GLOBAL tidb_distsql_scan_concurrency = 30;Modifies distributed query concurrency.
23Show ProcesslistSHOW PROCESSLIST;Inspects active SQL execution threads.
24Kill Connection ThreadKILL 83921;Terminates long-running query session.
25Check DDL JobsADMIN SHOW DDL JOBS;Inspects asynchronous schema changes.
26Cancel DDL JobADMIN CANCEL DDL JOBS 142;Stops active schema modification.
27Check Cluster DeadlocksSELECT * FROM information_schema.deadlocks;Inspects distributed transaction deadlocks.
28Flush Table PrivilegesFLUSH PRIVILEGES;Reloads access permission tables.
29Upgrade Clustertiup cluster upgrade prod-cluster v8.1.0Upgrades cluster software binaries.
30Cluster Scale Outtiup cluster scale-out prod-cluster scale-out-topo.yamlAdds compute/storage nodes.
31Scale In Clustertiup cluster scale-in prod-cluster --node 10.0.0.5:20160Safely removes node from cluster.
32Prune Clustertiup cluster prune prod-clusterCleans up decommissioned node binaries.
33Restart Componenttiup cluster restart prod-cluster -N 10.0.0.1:4000Restarts specific TiDB compute node.
34Reload Cluster Configtiup cluster reload prod-clusterReloads cluster configuration changes without restart.
35Check Cluster Audit Logtiup cluster audit prod-clusterInspects administrative CLI command history.
36Check Cluster Patch Versiontiup cluster patch prod-cluster patch.tar.gzApplies hotfix binary patch to cluster.
37Check PD Memberscurl http://127.0.0.1:2379/pd/api/v1/membersLists all Placement Driver nodes.
38Check PD Storescurl http://127.0.0.1:2379/pd/api/v1/storesLists all connected TiKV storage nodes.
39Check PD Regionscurl http://127.0.0.1:2379/pd/api/v1/regionsInspects total Raft region count and status.
40Check PD Hot Regionscurl http://127.0.0.1:2379/pd/api/v1/hotspot/regionsIdentifies read/write hotspot Raft regions.
41Set PD Leader Evicttiup ctl:v8.0.0 pd -u http://127.0.0.1:2379 scheduler add evict-leader-scheduler 1Evicts PD leader from specific node for maintenance.
42Remove PD Schedulertiup ctl:v8.0.0 pd -u http://127.0.0.1:2379 scheduler remove evict-leader-scheduler-1Removes eviction scheduler from PD.
43Check TiKV Store Statustiup ctl:v8.0.0 tikv --host 10.0.0.5:20160 storeInspects specific TiKV storage node metrics.
44Compact TiKV Rangetiup ctl:v8.0.0 tikv --host 10.0.0.5:20160 compact -d kvForces RocksDB SSTable compaction in TiKV.
45Check RocksDB MemorySELECT * FROM information_schema.tikv_auto_random;Inspects RocksDB block cache usage.
46Create Auto Random Primary KeyCREATE TABLE orders (id BIGINT AUTO_RANDOM PRIMARY KEY, val VARCHAR(50));Prevents write hotspotting on primary key.
47Create Clustered Index TableCREATE TABLE users (id INT, email VARCHAR(50), PRIMARY KEY(id) CLUSTERED);Enforces clustered index layout.
48Create Non-Clustered TableCREATE TABLE users (id INT, email VARCHAR(50), PRIMARY KEY(id) NONCLUSTERED);Enforces non-clustered table layout.
49Set Placement PolicyCREATE PLACEMENT POLICY east_policy PRIMARY_REGION="us-east-1" REGIONS="us-east-1,us-west-2";Provisions multi-region data placement policy.
50Apply Placement Policy to DBALTER DATABASE enterprise_db PLACEMENT POLICY=east_policy;Binds placement policy to database.
51Apply Placement Policy to TableALTER TABLE users PLACEMENT POLICY=east_policy;Binds placement policy to table.
52Show Placement PoliciesSHOW PLACEMENT POLICIES;Lists all configured placement policies.
53Drop Placement PolicyDROP PLACEMENT POLICY east_policy;Removes placement policy definition.
54Set Table Partition HashCREATE TABLE logs (id INT, dt DATE) PARTITION BY HASH(id) PARTITIONS 8;Provisions hash partitioned table.
55Set Table Partition RangeCREATE TABLE logs (id INT, val INT) PARTITION BY RANGE(val) (PARTITION p0 VALUES LESS THAN (1000));Provisions range partitioned table.
56Add Range PartitionALTER TABLE logs ADD PARTITION (PARTITION p1 VALUES LESS THAN (2000));Appends partition range.
57Drop Range PartitionALTER TABLE logs DROP PARTITION p0;Removes partition data instantly.
58Truncate Range PartitionALTER TABLE logs TRUNCATE PARTITION p1;Clears partition records.
59Check Partition RegionsSELECT * FROM information_schema.partitions WHERE table_name='logs';Inspects table partition metrics.
60Enable Dynamic Partition PruningSET SESSION tidb_partition_prune_mode = 'dynamic';Activates dynamic partition pruning for queries.
61Create Binding (SQL Plan)CREATE GLOBAL BINDING FOR SELECT * FROM users WHERE id = 1 USING SELECT /*+ USE_INDEX(users, idx_id) */ * FROM users WHERE id = 1;Forces query plan binding.
62Show Global BindingsSHOW GLOBAL BINDINGS;Lists active forced SQL query plan bindings.
63Drop Global BindingDROP GLOBAL BINDING FOR SELECT * FROM users WHERE id = 1;Removes forced SQL plan binding.
64Check Execution Plan CacheSELECT * FROM information_schema.plan_cache;Inspects prepared statement query plan cache.
65Flush Execution Plan CacheFLUSH PLAN_CACHE;Flushes prepared statement query plan cache.
66Check Resource Control PoolSELECT * FROM information_schema.resource_groups;Inspects CPU/IO resource control pools.
67Create Resource GroupCREATE RESOURCE GROUP rg_app RU_PER_SEC = 1000 BURSTABLE;Provisions resource control quota group.
68Alter Resource GroupALTER RESOURCE GROUP rg_app RU_PER_SEC = 2000;Updates resource control Request Units capacity.
69Assign User to Resource GroupALTER USER 'app_user'@'%' RESOURCE GROUP rg_app;Binds user account to resource control group.
70Drop Resource GroupDROP RESOURCE GROUP rg_app;Deletes resource control pool.
71Check Memory UsageSELECT * FROM information_schema.memory_usage;Pulls RAM usage breakdown across nodes.
72Set Memory Quota QuerySET SESSION tidb_mem_quota_query = 2147483648;Caps memory limit per query to 2GB.
73Enable OOM Action SpillSET SESSION tidb_mem_quota_query = 1073741824; SET SESSION tidb_enable_tmp_storage = ON;Spills query data to disk on OOM.
74Check Global VariablesSHOW GLOBAL VARIABLES LIKE 'tidb%';Inspects global TiDB cluster configuration.
75Set Index MergeSET SESSION tidb_enable_index_merge = ON;Activates index merge scan optimization.
76Set Async CommitSET GLOBAL tidb_enable_async_commit = ON;Enables two-phase async commit for speed.
77Set 1PC CommitSET GLOBAL tidb_enable_1pc = ON;Enables single-phase commit for non-distributed transactions.
78Create SequenceCREATE SEQUENCE user_seq START WITH 1 INCREMENT BY 1;Provisions distributed sequence object.
79Get Sequence Next ValueSELECT NEXTVAL(user_seq);Fetches next value from distributed sequence.
80Drop SequenceDROP SEQUENCE user_seq;Deletes sequence object.
81Check Metadata LockSELECT * FROM information_schema.metadata_locks;Inspects blocking DDL metadata locks.
82Check Table ConstraintsSELECT * FROM information_schema.table_constraints WHERE table_name='users';Lists unique and check constraints.
83Check Column Character SetSHOW FULL COLUMNS FROM users;Inspects column collations and character sets.
84Check Table Storage EngineSELECT table_name, engine FROM information_schema.tables WHERE table_schema='enterprise_db';Inspects table engines (InnoDB/TiFlash).
85Set TiFlash Pipeline EnforcementSET SESSION tidb_allow_mpp = ON;Enforces TiFlash Massively Parallel Processing mode.
86Check TiFlash Query ProgressSELECT * FROM information_schema.tiflash_tables;Monitors TiFlash columnar query execution.
87Create Expression IndexALTER TABLE users ADD INDEX idx_lower_email ((LOWER(email)));Provisions expression index.
88Create Invisible IndexALTER TABLE users ADD INDEX idx_status (status) INVISIBLE;Provisions hidden index for testing.
89Alter Index VisibleALTER TABLE users ALTER INDEX idx_status VISIBLE;Restores index visibility to query optimizer.
90Compact Table StorageADMIN COMPACT TABLE users;Triggers background storage defragmentation.
91Check DDL HistoryADMIN SHOW DDL JOBS 10;Inspects completed asynchronous DDL jobs.
92Recover Table (Flashback)RECOVER TABLE users;Recovers dropped table before GC threshold.
93Flashback DatabaseFLASHBACK DATABASE enterprise_db TO TIMESTAMP '2026-07-01 10:00:00';Reverts database to historical point in time.
94Check GC Life TimeSELECT * FROM mysql.tidb WHERE variable_name='tikv_gc_life_time';Inspects garbage collection history window.
95Set GC Life TimeUPDATE mysql.tidb SET variable_value='24h' WHERE variable_name='tikv_gc_life_time';Extends GC retention window for time travel.
96Check Cluster VersionSELECT TIDB_VERSION();Outputs detailed TiDB build commit and release version.
97Check DB Health HTTPcurl http://127.0.0.1:10080/statusQueries HTTP status endpoint for TiDB SQL node status.
98Check Metrics Prometheus HTTPcurl http://127.0.0.1:10080/metricsPulls Prometheus telemetry metrics from TiDB SQL node.
99Show Create Table DDLSHOW CREATE TABLE users;Outputs exact DDL syntax used to build table.
100Show Create Database DDLSHOW CREATE DATABASE enterprise_db;Outputs exact DDL syntax used to build database.
101Scale In Clustertiup cluster scale-in prod-cluster --node 10.0.0.5:20160Safely removes node from cluster.
102Prune Clustertiup cluster prune prod-clusterCleans up decommissioned node binaries.
103Restart Componenttiup cluster restart prod-cluster -N 10.0.0.1:4000Restarts specific TiDB compute node.
104Reload Cluster Configtiup cluster reload prod-clusterReloads cluster configuration changes without restart.
105Check Cluster Audit Logtiup cluster audit prod-clusterInspects administrative CLI command history.
106Check PD Memberscurl http://127.0.0.1:2379/pd/api/v1/membersLists all Placement Driver nodes.
107Check PD Storescurl http://127.0.0.1:2379/pd/api/v1/storesLists all connected TiKV storage nodes.
108Check PD Regionscurl http://127.0.0.1:2379/pd/api/v1/regionsInspects total Raft region count and status.
109Check PD Hot Regionscurl http://127.0.0.1:2379/pd/api/v1/hotspot/regionsIdentifies read/write hotspot Raft regions.
110Compact TiKV Rangetiup ctl:v8.0.0 tikv --host 10.0.0.5:20160 compact -d kvForces RocksDB SSTable compaction in TiKV.
111Create Auto Random Primary KeyCREATE TABLE orders (id BIGINT AUTO_RANDOM PRIMARY KEY, val VARCHAR(50));Prevents write hotspotting on primary key.
112Create Clustered Index TableCREATE TABLE users (id INT, email VARCHAR(50), PRIMARY KEY(id) CLUSTERED);Enforces clustered index layout.
113Set Placement PolicyCREATE PLACEMENT POLICY east_policy PRIMARY_REGION="us-east-1" REGIONS="us-east-1,us-west-2";Provisions multi-region data placement policy.
114Apply Placement Policy to TableALTER TABLE users PLACEMENT POLICY=east_policy;Binds placement policy to table.
115Set Table Partition HashCREATE TABLE logs (id INT, dt DATE) PARTITION BY HASH(id) PARTITIONS 8;Provisions hash partitioned table.
116Add Range PartitionALTER TABLE logs ADD PARTITION (PARTITION p1 VALUES LESS THAN (2000));Appends partition range.
117Drop Range PartitionALTER TABLE logs DROP PARTITION p0;Removes partition data instantly.
118Enable Dynamic Partition PruningSET SESSION tidb_partition_prune_mode = 'dynamic';Activates dynamic partition pruning for queries.
119Create Global BindingCREATE GLOBAL BINDING FOR SELECT * FROM users WHERE id = 1 USING SELECT /*+ USE_INDEX(users, idx_id) */ * FROM users WHERE id = 1;Forces query plan binding.
120Show Global BindingsSHOW GLOBAL BINDINGS;Lists active forced SQL query plan bindings.
121Check Resource GroupSELECT * FROM information_schema.resource_groups;Inspects CPU/IO resource control pools.
122Create Resource GroupCREATE RESOURCE GROUP rg_app RU_PER_SEC = 1000 BURSTABLE;Provisions resource control quota group.
123Assign User to Resource GroupALTER USER 'app_user'@'%' RESOURCE GROUP rg_app;Binds user account to resource control group.
124Set Memory Quota QuerySET SESSION tidb_mem_quota_query = 2147483648;Caps memory limit per query to 2GB.
125Check Metadata LocksSELECT * FROM information_schema.metadata_locks;Inspects blocking DDL metadata locks.
126Set TiFlash Pipeline ModeSET SESSION tidb_allow_mpp = ON;Enforces TiFlash Massively Parallel Processing mode.
127Check TiFlash StatusSELECT * FROM information_schema.tiflash_tables;Monitors TiFlash columnar query execution.
128Recover Table (Flashback)RECOVER TABLE users;Recovers dropped table before GC threshold.
129Check Cluster VersionSELECT TIDB_VERSION();Outputs detailed TiDB build commit and release version.
130Check DB Health HTTPcurl http://127.0.0.1:10080/statusQueries HTTP status endpoint for TiDB SQL node status.
131Check TiDB Node IDSELECT @@hostname, @@port;Outputs TiDB SQL node hostname and port.
132Check Active Connections CountSELECT count(*) FROM information_schema.processlist;Counts total connected client sessions.
133Check Running SQL Queries CountSELECT count(*) FROM information_schema.processlist WHERE command <> 'Sleep';Counts non-idle running SQL queries.
134Check Slow Query Log ThresholdSHOW VARIABLES LIKE 'tidb_slow_log_threshold';Inspects slow query logging threshold (ms).
135Set Slow Query Log ThresholdSET GLOBAL tidb_slow_log_threshold = 300;Logs queries taking longer than 300ms.
136Check Expensive Queries Info SchemaSELECT * FROM information_schema.tidb_trx ORDER BY duration DESC LIMIT 10;Inspects longest running active transactions.
137Check Lock Waits Info SchemaSELECT * FROM information_schema.data_lock_waits;Inspects transaction lock wait contention.
138Check Deadlock History Info SchemaSELECT * FROM information_schema.deadlocks ORDER BY occur_time DESC LIMIT 10;Inspects historical deadlock events.
139Check TiKV Region Count per StoreSELECT store_id, count(*) AS region_count FROM information_schema.tikv_region_status GROUP BY store_id;Calculates region distribution across TiKV stores.
140Check TiKV Region Size StatsSELECT store_id, min(approximate_size), avg(approximate_size), max(approximate_size) FROM information_schema.tikv_region_status GROUP BY store_id;Calculates region byte size statistics.
141Check TiKV Peer DistributionSELECT store_id, count(*) AS peer_count FROM information_schema.tikv_region_peers GROUP BY store_id;Calculates Raft peer distribution across storage nodes.
142Check TiKV Leader DistributionSELECT store_id, count(*) AS leader_count FROM information_schema.tikv_region_status WHERE is_leader = 1 GROUP BY store_id;Calculates Raft leader distribution across storage nodes.
143Check TiKV Offline StoresSELECT * FROM information_schema.tikv_store_status WHERE state_name <> 'Up';Identifies offline or degraded TiKV storage nodes.
144Check TiFlash Replica Sync StatusSELECT table_name, replica_count, progress FROM information_schema.tiflash_replica;Monitors progress of columnar data sync.
145Check TiFlash Node ListSELECT * FROM information_schema.tiflash_tables;Lists all tables replicated to TiFlash columnar engine.
146Check Placement Driver Leader HostSELECT * FROM information_schema.pd_status;Inspects active Placement Driver leader node IP and port.
147Check Auto Increment StepSHOW VARIABLES LIKE 'auto_increment_increment';Inspects auto-increment step value.
148Check Auto Random ID BitsSHOW VARIABLES LIKE 'tidb_auto_random_explicit_insert';Inspects auto-random ID generation settings.
149Check DDL Execution StatusADMIN SHOW DDL;Inspects current DDL execution phase and schema version.
150Check DDL Job HistoryADMIN SHOW DDL JOBS 20;Lists last 20 executed asynchronous DDL jobs.
151Check DDL Job DetailsADMIN SHOW DDL JOB QUERIES 142;Outputs full SQL statement for specific DDL job ID.
152Cancel Running DDL JobADMIN CANCEL DDL JOBS 142;Aborts active running DDL schema modification job.
153Check Execution Plan Cache StatusSHOW VARIABLES LIKE 'tidb_enable_plan_cache';Inspects query execution plan cache status.
154Set Enable Execution Plan CacheSET GLOBAL tidb_enable_plan_cache = ON;Enables execution plan caching for prepared statements.
155Check Execution Plan Cache MemorySELECT * FROM information_schema.plan_cache;Inspects cached execution plans in RAM.
156Flush Execution Plan CacheFLUSH PLAN_CACHE;Flushes prepared statement execution plan cache.
157Check Resource Group AllocationsSELECT * FROM information_schema.resource_groups;Lists resource control groups and Request Units (RU) quotas.
158Check Resource Group User MappingSELECT user, inline_role, resource_group FROM mysql.user;Inspects resource groups assigned to user accounts.
159Check Resource Group Usage MetricsSELECT * FROM information_schema.resource_group_usage;Monitors RU consumption per resource group.
160Check Memory Usage per QuerySELECT * FROM information_schema.memory_usage_ops;Pulls memory allocation breakdown across SQL operators.
161Check Memory Usage per SessionSELECT * FROM information_schema.processlist ORDER BY memory DESC LIMIT 10;Finds client sessions consuming highest RAM.
162Check Temp Storage DirectorySHOW VARIABLES LIKE 'tidb_tmp_storage_path';Inspects disk directory path for query spill files.
163Check Temp Storage Max SizeSHOW VARIABLES LIKE 'tidb_tmp_storage_quota';Inspects disk spill storage quota limit.
164Check Global System VariablesSHOW GLOBAL VARIABLES LIKE 'tidb_%';Lists all global TiDB cluster configuration variables.
165Check Session System VariablesSHOW SESSION VARIABLES LIKE 'tidb_%';Lists all active session TiDB configuration variables.
166Check Async Commit EnabledSHOW VARIABLES LIKE 'tidb_enable_async_commit';Verifies whether two-phase async commit is enabled.
167Check 1PC Commit EnabledSHOW VARIABLES LIKE 'tidb_enable_1pc';Verifies whether single-phase commit is enabled.
168Check Metadata Lock TimeoutSHOW VARIABLES LIKE 'tidb_metadata_lock_timeout';Inspects DDL metadata lock wait timeout seconds.
169Check Table Partition Pruning ModeSHOW VARIABLES LIKE 'tidb_partition_prune_mode';Inspects partition pruning mode (static vs dynamic).
170Check Table Storage Engine ListSHOW ENGINES;Lists supported table storage engines (InnoDB, TiFlash).
171Check Table Constraints MetadataSELECT * FROM information_schema.table_constraints WHERE table_schema='enterprise_db';Lists all constraints in schema.
172Check Table Column CollationsSHOW FULL COLUMNS FROM users;Inspects column collations and character set encodings.
173Check Sequence Current ValueSELECT CURRENTVAL(user_seq);Returns last generated value from distributed sequence.
174Check Sequence Next ValueSELECT NEXTVAL(user_seq);Generates next value from distributed sequence.
175Check Sequence DetailsSHOW CREATE SEQUENCE user_seq;Outputs DDL syntax used to build sequence.
176Check GC Life Time SettingSELECT * FROM mysql.tidb WHERE variable_name='tikv_gc_life_time';Inspects time travel garbage collection retention window.
177Set GC Life Time WindowUPDATE mysql.tidb SET variable_value='48h' WHERE variable_name='tikv_gc_life_time';Extends time travel window to 48 hours.
178Check GC Safe Point TimestampSELECT * FROM mysql.tidb WHERE variable_name='tikv_gc_safe_point';Inspects oldest timestamp safe for time travel reads.
179Check TiDB Server Version SQLSELECT TIDB_VERSION();Outputs detailed TiDB build release and git commit hash.
180Check Connection Ping SQLSELECT 1;Simple connection ping test returning 1.
181Check Current User AccountSELECT CURRENT_USER();Returns current execution user account.
182Check Current Database NameSELECT DATABASE();Returns active selected database name.
183Check System TimestampSELECT NOW();Returns current server timestamp.
184Check Connection IDSELECT CONNECTION_ID();Returns current connection thread ID.
185Check HTTP Health Status REST APIcurl http://127.0.0.1:10080/statusQueries HTTP status endpoint for SQL node health.
186Check Prometheus Metrics REST APIcurl http://127.0.0.1:10080/metricsPulls Prometheus metrics endpoint from SQL node.
187Check PD HTTP Leader REST APIcurl http://127.0.0.1:2379/pd/api/v1/leaderQueries Placement Driver HTTP REST endpoint for leader status.
188Check PD HTTP Members REST APIcurl http://127.0.0.1:2379/pd/api/v1/membersQueries Placement Driver HTTP REST endpoint for member list.
189Check TiUP Cluster List CLItiup cluster listLists all deployed TiUP clusters.
190Check TiUP Cluster Display CLItiup cluster display prod-clusterOutputs status table for all cluster nodes via TiUP CLI.
191Check Table Column Types MetadataSELECT column_name, data_type FROM information_schema.columns WHERE table_name = 'users';Inspects column data types.
192Check Table Column Nullability MetadataSELECT column_name, is_nullable FROM information_schema.columns WHERE table_name = 'users';Inspects column nullability.
193Check Table Primary Key MetadataSELECT column_name FROM information_schema.key_column_usage WHERE table_name = 'users';Inspects primary key columns.
194Check Table Unique Constraints MetadataSELECT constraint_name FROM information_schema.table_constraints WHERE table_name = 'users' AND constraint_type = 'UNIQUE';Lists unique constraints.
195Check Table Foreign Keys MetadataSELECT constraint_name FROM information_schema.table_constraints WHERE table_name = 'users' AND constraint_type = 'FOREIGN KEY';Lists foreign keys.
196Check Table Indexes MetadataSELECT index_name FROM information_schema.statistics WHERE table_name = 'users';Lists indexes on table.
197Check Table Storage Engine TypeSELECT engine FROM information_schema.tables WHERE table_name = 'users';Inspects table storage engine.
198Check Table Row Count EstimationSELECT table_rows FROM information_schema.tables WHERE table_name = 'users';Inspects estimated table row count.
199Check Database Table CountSELECT count(*) FROM information_schema.tables WHERE table_schema = 'enterprise_db';Counts total tables in database.
200Check Database View CountSELECT count(*) FROM information_schema.views WHERE table_schema = 'enterprise_db';Counts total views in database.

Amazon Aurora OLTP

Technical Architecture & Overview

Amazon Aurora is a cloud-native relational database service compatible with MySQL and PostgreSQL. It decouples compute from storage, utilizing a shared distributed storage volume replicated across 3 Availability Zones with 6-way storage copy persistence, fast auto-scaling, and Serverless v2 capacity management.

Primary Use Cases: High-performance enterprise OLTP, mission-critical cloud applications, serverless auto-scaling microservices, and multi-region global databases.

Core Components: Compute Nodes (Writer & Read Replicas), Shared Distributed Log-Structured Storage Volume, Aurora Serverless v2 Controller, and Global Database Replication.

Exhaustive 200-command administrative, diagnostic, and operational matrix for Amazon Aurora.

#Operation / ActionAWS CLI / SQL SyntaxDescription
1Create Aurora Clusteraws rds create-db-cluster --db-cluster-identifier prod-aurora --engine aurora-postgresql --master-username postgres --master-user-password SecurePass123Provisions managed Aurora cluster.
2Delete Aurora Clusteraws rds delete-db-cluster --db-cluster-identifier prod-aurora --skip-final-snapshotDecommissions Aurora cluster.
3Create Aurora Reader Instanceaws rds create-db-instance --db-instance-identifier aurora-read-1 --db-cluster-identifier prod-aurora --db-instance-class db.r6g.xlarge --engine aurora-postgresqlAdds read-replica compute instance.
4Failover Aurora Clusteraws rds failover-db-cluster --db-cluster-identifier prod-aurora --target-db-instance-identifier aurora-read-1Triggers manual cluster failover.
5Describe Aurora Clustersaws rds describe-db-clusters --db-cluster-identifier prod-auroraInspects cluster status and endpoints.
6Create DB Subnet Groupaws rds create-db-subnet-group --db-subnet-group-name aurora-subnets --subnet-ids subnet-123 subnet-456Configures VPC subnet grouping.
7Create Parameter Groupaws rds create-db-parameter-group --db-parameter-group-name custom-aurora-pg --family aurora-postgresql15Provisions custom tuning parameters.
8Modify Parameter Groupaws rds modify-db-parameter-group --db-parameter-group-name custom-aurora-pg --parameters "ParameterName=max_connections,ParameterValue=1000,ApplyMethod=immediate"Updates runtime server parameters.
9Create Snapshotaws rds create-db-cluster-snapshot --db-cluster-identifier prod-aurora --db-cluster-snapshot-identifier aurora-backup-01Takes manual storage snapshot.
10Restore from Snapshotaws rds restore-db-cluster-from-snapshot --db-cluster-identifier prod-aurora-restored --snapshot-identifier aurora-backup-01 --engine aurora-postgresqlRestores cluster from backup snapshot.
11Modify Cluster Scalingaws rds modify-db-cluster --db-cluster-identifier prod-aurora --scaling-configuration MinCapacity=2,MaxCapacity=32,SecondsUntilAutoPause=300Configures Aurora Serverless v2.
12Connect via PostgreSQL Clientpsql -h prod-aurora.cluster-123.us-east-1.rds.amazonaws.com -U postgres -d enterprise_dbConnects to Aurora cluster endpoint.
13Check Aurora Storage VolumeSELECT aurora_version();Inspects storage engine version.
14Monitor Replication LagSELECT * FROM pg_stat_replication;Checks read replica replication lag.
15Enable Performance Insightsaws rds modify-db-instance --db-instance-identifier aurora-read-1 --enable-performance-insights --performance-insights-retention-period 7Activates deep query telemetry.
16Reboot Instanceaws rds reboot-db-instance --db-instance-identifier aurora-read-1Restarts compute node instance.
17Add Tags to Clusteraws rds add-tags-to-resource --resource-name arn:aws:rds:... --tags Key=Environment,Value=ProductionApplies cloud asset tags.
18List Engine Versionsaws rds describe-db-engine-versions --engine aurora-postgresqlInspects available patch versions.
19Create Global Databaseaws rds create-global-cluster --global-cluster-identifier prod-global --source-db-cluster-identifier arn:aws:rds:...Provisions cross-region global cluster.
20Remove Secondary Global Clusteraws rds remove-from-global-cluster --global-cluster-identifier prod-global --db-cluster-identifier arn:aws:rds:...Detaches cross-region replica.
21Failover Global Clusteraws rds failover-global-cluster --global-cluster-identifier prod-global --target-db-cluster-identifier arn:aws:rds:...Triggers cross-region disaster recovery.
22Describe Event Subscriptionsaws rds describe-event-subscriptionsInspects CloudWatch notification hooks.
23Create Event Subscriptionaws rds create-event-subscription --subscription-name aurora-alerts --sns-topic-arn arn:aws:sns:... --source-type db-clusterConfigures database alert notifications.
24Export to S3 (Audit Logs)aws rds modify-db-cluster --db-cluster-identifier prod-aurora --cloudwatch-logs-export-credentials-arn ... --cloudwatch-logs-export-configuration '{"EnableLogTypes":["postgresql"]}'Exports logs to CloudWatch/S3.
25Describe DB Log Filesaws rds describe-db-log-files --db-instance-identifier aurora-read-1Lists diagnostic log file artifacts.
26Download Log File Segmentaws rds download-db-log-file-portion --db-instance-identifier aurora-read-1 --log-file-name error/postgresql.log --output textDownloads error log contents.
27Modify Security Groupsaws rds modify-db-instance --db-instance-identifier aurora-read-1 --vpc-security-group-ids sg-98765432Updates firewall security group rules.
28Modify Maintenance Windowaws rds modify-db-cluster --db-cluster-identifier prod-aurora --preferred-maintenance-window sun:04:00-sun:05:00Schedules automated patch maintenance.
29Describe Reserved Instancesaws rds describe-reserved-db-instancesInspects database cost-saving discounts.
30Get CloudWatch Metricsaws cloudwatch get-metric-data --metric-data-queries ...Pulls infrastructure CPU and memory metrics.
31Start Clusteraws rds start-db-cluster --db-cluster-identifier prod-auroraBoots stopped Aurora cluster.
32Stop Clusteraws rds stop-db-cluster --db-cluster-identifier prod-auroraHalts compute nodes for cost savings.
33Promote Read Replicaaws rds promote-read-replica --db-instance-identifier aurora-read-1Promotes reader instance to standalone cluster.
34Create Cluster Parameter Groupaws rds create-db-cluster-parameter-group --db-cluster-parameter-group-name custom-cluster-pg --db-parameter-group-family aurora-postgresql15Provisions cluster-level parameter group.
35Modify Cluster Parameter Groupaws rds modify-db-cluster-parameter-group --db-cluster-parameter-group-name custom-cluster-pg --parameters "ParameterName=rds.force_ssl,ParameterValue=1,ApplyMethod=immediate"Enforces SSL connections cluster-wide.
36Describe Cluster Parametersaws rds describe-db-cluster-parameters --db-cluster-parameter-group-name custom-cluster-pgInspects cluster parameter settings.
37Reset Cluster Parameter Groupaws rds reset-db-cluster-parameter-group --db-cluster-parameter-group-name custom-cluster-pg --reset-all-parametersRestores parameter group to defaults.
38Modify DB Instance Classaws rds modify-db-instance --db-instance-identifier aurora-read-1 --db-instance-class db.r6g.2xlarge --apply-immediatelyScales compute instance sizing.
39Enable Deletion Protectionaws rds modify-db-cluster --db-cluster-identifier prod-aurora --deletion-protectionPrevents accidental cluster deletion.
40Disable Deletion Protectionaws rds modify-db-cluster --db-cluster-identifier prod-aurora --no-deletion-protectionAllows cluster deletion.
41Enable IAM Authenticationaws rds modify-db-cluster --db-cluster-identifier prod-aurora --enable-iam-database-authenticationEnables IAM token database login.
42Disable IAM Authenticationaws rds modify-db-cluster --db-cluster-identifier prod-aurora --no-enable-iam-database-authenticationDisables IAM token database login.
43Create Auto Scaling Policyaws application-autoscaling register-scalable-target --service-namespace rds --resource-id cluster:prod-aurora --scalable-dimension rds:subgrid:ReadReplicaCount --min-capacity 1 --max-capacity 15Configures Auto Scaling target.
44Put Scaling Policy Triggeraws application-autoscaling put-scaling-policy --policy-name cpu-auto-scaling --service-namespace rds --resource-id cluster:prod-aurora --scalable-dimension rds:subgrid:ReadReplicaCount --policy-type TargetTrackingScaling --target-tracking-scaling-policy-configuration ...Enforces CPU target tracking for replicas.
45Delete Auto Scaling Policyaws application-autoscaling deregister-scalable-target --service-namespace rds --resource-id cluster:prod-aurora --scalable-dimension rds:subgrid:ReadReplicaCountRemoves auto-scaling target.
46Copy Cluster Snapshotaws rds copy-db-cluster-snapshot --source-db-cluster-snapshot-identifier aurora-backup-01 --target-db-cluster-snapshot-identifier aurora-backup-01-copyDuplicates cluster snapshot.
47Share Snapshot with Accountaws rds modify-db-cluster-snapshot-attribute --db-cluster-snapshot-identifier aurora-backup-01 --attribute-name restore --values-to-add 123456789012Shares snapshot with external AWS account.
48Delete Cluster Snapshotaws rds delete-db-cluster-snapshot --db-cluster-snapshot-identifier aurora-backup-01Deletes storage snapshot.
49Restore To Point In Timeaws rds restore-db-cluster-to-point-in-time --target-db-cluster-identifier prod-aurora-pitr --source-db-cluster-identifier prod-aurora --restore-to-time 2026-07-29T12:00:00ZExecutes point-in-time recovery.
50Check Backtrack Windowaws rds describe-db-clusters --db-cluster-identifier prod-aurora --query "DBClusters[0].BacktrackWindow"Inspects active backtrack time window.
51Backtrack DB Clusteraws rds backtrack-db-cluster --db-cluster-identifier prod-aurora --backtrack-to 2026-07-29T10:00:00ZRewinds cluster storage in minutes via Aurora Backtrack.
52Create Custom Endpointaws rds create-db-cluster-endpoint --db-cluster-identifier prod-aurora --db-cluster-endpoint-identifier analytics-endpoint --endpoint-type READER --static-members aurora-read-1 aurora-read-2Provisions custom reader endpoint.
53Describe Custom Endpointsaws rds describe-db-cluster-endpoints --db-cluster-identifier prod-auroraInspects custom endpoint status and members.
54Modify Custom Endpointaws rds modify-db-cluster-endpoint --db-cluster-endpoint-identifier analytics-endpoint --static-members aurora-read-1 aurora-read-2 aurora-read-3Adds replica to custom endpoint.
55Delete Custom Endpointaws rds delete-db-cluster-endpoint --db-cluster-endpoint-identifier analytics-endpointDeletes custom endpoint.
56Create Aurora Cloneaws rds restore-db-cluster-to-point-in-time --target-db-cluster-identifier prod-aurora-clone --source-db-cluster-identifier prod-aurora --restore-type copy-on-writeCreates fast zero-copy storage clone.
57Check Aurora Limit Quotasaws rds describe-account-attributesInspects account quotas for instances and clusters.
58Check Aurora Storage UsageSELECT sum(aurora_volume_num_pages()) * 16384 / 1024 / 1024 / 1024 AS storage_gb;Calculates exact allocated Aurora storage.
59Check Aurora Replica Lag SQLSELECT server_id, slot_name, active, round(aurora_replica_status()) FROM aurora_replica_status();Queries Aurora internal replica lag SQL function.
60Check Aurora Version SQLSELECT aurora_version();Outputs running Aurora engine version string.
61Enable Aurora PostgreSQL PGVectorCREATE EXTENSION IF NOT EXISTS vector;Provisions pgvector extension in Aurora PG.
62Enable Aurora PostGISCREATE EXTENSION IF NOT EXISTS postgis;Provisions spatial PostGIS extension in Aurora PG.
63Enable Aurora AWS S3 ExportCREATE EXTENSION IF NOT EXISTS aws_s3;Provisions AWS S3 export extension.
64Export Table to S3 SQLSELECT * FROM aws_s3.query_export_to_s3('SELECT * FROM users', aws_commons.create_s3_uri('my-bucket', 'export.csv', 'us-east-1'));Directly exports SQL query output to S3.
65Enable Aurora AWS LambdaCREATE EXTENSION IF NOT EXISTS aws_lambda;Provisions AWS Lambda trigger extension.
66Invoke Lambda Function SQLSELECT * FROM aws_lambda.invoke(aws_commons.create_lambda_function_arn('arn:aws:lambda:...'), '{"key":"val"}'::json);Calls Lambda function directly from SQL.
67Check Performance Insights Top Locksaws pi get-resource-metrics --service-type RDS --identifier db-12345 --metric-queries ...Queries Performance Insights for lock waits.
68Check Performance Insights Top Queriesaws pi get-resource-metrics --service-type RDS --identifier db-12345 --metric-queries ...Queries Performance Insights for top SQL load.
69List DB Subnet Groupsaws rds describe-db-subnet-groupsLists all VPC DB subnet groups.
70Delete DB Subnet Groupaws rds delete-db-subnet-group --db-subnet-group-name aurora-subnetsDeletes VPC DB subnet group.
71Modify DB Subnet Groupaws rds modify-db-subnet-group --db-subnet-group-name aurora-subnets --subnet-ids subnet-123 subnet-456 subnet-789Updates subnets in subnet group.
72List Parameter Groupsaws rds describe-db-parameter-groupsLists all DB instance parameter groups.
73Delete Parameter Groupaws rds delete-db-parameter-group --db-parameter-group-name custom-aurora-pgDeletes instance parameter group.
74List Cluster Parameter Groupsaws rds describe-db-cluster-parameter-groupsLists all DB cluster parameter groups.
75Delete Cluster Parameter Groupaws rds delete-db-cluster-parameter-group --db-cluster-parameter-group-name custom-cluster-pgDeletes cluster parameter group.
76List Option Groupsaws rds describe-option-groupsLists all RDS option groups.
77Create Option Groupaws rds create-option-group --option-group-name custom-og --engine-name aurora-mysql --major-engine-version 8.0 --option-group-description "Custom options"Provisions option group.
78Delete Option Groupaws rds delete-option-group --option-group-name custom-ogDeletes option group.
79List Event Categoriesaws rds describe-event-categoriesLists supported notification event categories.
80Delete Event Subscriptionaws rds delete-event-subscription --subscription-name aurora-alertsRemoves event notification subscription.
81Modify Event Subscriptionaws rds modify-event-subscription --subscription-name aurora-alerts --enabledActivates event notification subscription.
82List Pending Maintenance Actionsaws rds describe-pending-maintenance-actionsInspects required OS/database security patches.
83Apply Pending Maintenanceaws rds apply-pending-maintenance-action --resource-identifier arn:aws:rds:... --apply-action system-update --opt-in-type immediateApplies pending security patch immediately.
84Describe Global Clustersaws rds describe-global-clustersLists all cross-region global databases.
85Failover Global Cluster Automatedaws rds failover-global-cluster --global-cluster-identifier prod-global --target-db-cluster-identifier arn:aws:rds:...Executes managed global cluster failover.
86Modify Global Clusteraws rds modify-global-cluster --global-cluster-identifier prod-global --deletion-protectionEnables deletion protection on global database.
87Delete Global Clusteraws rds delete-global-cluster --global-cluster-identifier prod-globalDeletes empty global database container.
88List DB Security Groupsaws rds describe-db-security-groupsLists EC2-Classic DB security groups.
89List Certificatesaws rds describe-certificatesLists active CA SSL/TLS certificates for RDS.
90Modify Certificate Identifieraws rds modify-db-instance --db-instance-identifier aurora-read-1 --ca-certificate-identifier rds-ca-rsa4096-g1Rotates SSL/TLS CA certificate.
91Check Aurora Database Activity Streamsaws rds start-activity-stream --resource-arn arn:aws:rds:... --mode async --kms-key-id arn:aws:kms:...Activates database activity streaming to Kinesis.
92Stop Database Activity Streamaws rds stop-activity-stream --resource-arn arn:aws:rds:...Halts database activity streaming.
93Describe Database Activity Streamsaws rds describe-db-clusters --db-cluster-identifier prod-aurora --query "DBClusters[0].ActivityStreamStatus"Inspects status of activity stream.
94Check Blue/Green Deploymentsaws rds describe-blue-green-deploymentsLists active staging blue/green deployment environments.
95Create Blue/Green Deploymentaws rds create-blue-green-deployment --blue-green-deployment-name bg-upgrade --source arn:aws:rds:... --target-engine-version 15.4Provisions isolated staging environment for zero-downtime upgrade.
96Switch Blue/Green Deploymentaws rds switchover-blue-green-deployment --blue-green-deployment-identifier bgd-12345Switches production traffic to green environment.
97Delete Blue/Green Deploymentaws rds delete-blue-green-deployment --blue-green-deployment-identifier bgd-12345 --delete-targetDeletes staging blue/green environment.
98Reboot Clusteraws rds reboot-db-cluster --db-cluster-identifier prod-auroraRestarts all nodes in Aurora cluster.
99Check Database Engine Version SQLSELECT version();Outputs detailed database engine build version.
100Check Active Cluster ConnectionsSELECT count(*) FROM pg_stat_activity;Counts active client connection sessions.
101Start Clusteraws rds start-db-cluster --db-cluster-identifier prod-auroraBoots stopped Aurora cluster.
102Stop Clusteraws rds stop-db-cluster --db-cluster-identifier prod-auroraHalts compute nodes for cost savings.
103Promote Read Replicaaws rds promote-read-replica --db-instance-identifier aurora-read-1Promotes reader instance to standalone cluster.
104Create Cluster Parameter Groupaws rds create-db-cluster-parameter-group --db-cluster-parameter-group-name custom-cluster-pg --db-parameter-group-family aurora-postgresql15Provisions cluster-level parameter group.
105Modify Cluster Parameter Groupaws rds modify-db-cluster-parameter-group --db-cluster-parameter-group-name custom-cluster-pg --parameters "ParameterName=rds.force_ssl,ParameterValue=1,ApplyMethod=immediate"Enforces SSL connections cluster-wide.
106Describe Cluster Parametersaws rds describe-db-cluster-parameters --db-cluster-parameter-group-name custom-cluster-pgInspects cluster parameter settings.
107Enable Deletion Protectionaws rds modify-db-cluster --db-cluster-identifier prod-aurora --deletion-protectionPrevents accidental cluster deletion.
108Enable IAM Authenticationaws rds modify-db-cluster --db-cluster-identifier prod-aurora --enable-iam-database-authenticationEnables IAM token database login.
109Create Auto Scaling Policyaws application-autoscaling register-scalable-target --service-namespace rds --resource-id cluster:prod-aurora --scalable-dimension rds:subgrid:ReadReplicaCount --min-capacity 1 --max-capacity 15Configures Auto Scaling target.
110Copy Cluster Snapshotaws rds copy-db-cluster-snapshot --source-db-cluster-snapshot-identifier aurora-backup-01 --target-db-cluster-snapshot-identifier aurora-backup-01-copyDuplicates cluster snapshot.
111Restore To Point In Timeaws rds restore-db-cluster-to-point-in-time --target-db-cluster-identifier prod-aurora-pitr --source-db-cluster-identifier prod-aurora --restore-to-time 2026-07-29T12:00:00ZExecutes point-in-time recovery.
112Backtrack DB Clusteraws rds backtrack-db-cluster --db-cluster-identifier prod-aurora --backtrack-to 2026-07-29T10:00:00ZRewinds cluster storage in minutes via Aurora Backtrack.
113Create Custom Endpointaws rds create-db-cluster-endpoint --db-cluster-identifier prod-aurora --db-cluster-endpoint-identifier analytics-endpoint --endpoint-type READER --static-members aurora-read-1 aurora-read-2Provisions custom reader endpoint.
114Create Aurora Cloneaws rds restore-db-cluster-to-point-in-time --target-db-cluster-identifier prod-aurora-clone --source-db-cluster-identifier prod-aurora --restore-type copy-on-writeCreates fast zero-copy storage clone.
115Check Aurora Replica Lag SQLSELECT server_id, slot_name, active, round(aurora_replica_status()) FROM aurora_replica_status();Queries Aurora internal replica lag SQL function.
116Enable Aurora PGVectorCREATE EXTENSION IF NOT EXISTS vector;Provisions pgvector extension in Aurora PG.
117Enable Aurora PostGISCREATE EXTENSION IF NOT EXISTS postgis;Provisions spatial PostGIS extension in Aurora PG.
118Export Table to S3 SQLSELECT * FROM aws_s3.query_export_to_s3('SELECT * FROM users', aws_commons.create_s3_uri('my-bucket', 'export.csv', 'us-east-1'));Directly exports SQL query output to S3.
119Invoke Lambda Function SQLSELECT * FROM aws_lambda.invoke(aws_commons.create_lambda_function_arn('arn:aws:lambda:...'), '{"key":"val"}'::json);Calls Lambda function directly from SQL.
120Check Performance Insights Top Queriesaws pi get-resource-metrics --service-type RDS --identifier db-12345 --metric-queries ...Queries Performance Insights for top SQL load.
121Apply Pending Maintenanceaws rds apply-pending-maintenance-action --resource-identifier arn:aws:rds:... --apply-action system-update --opt-in-type immediateApplies pending security patch immediately.
122Failover Global Clusteraws rds failover-global-cluster --global-cluster-identifier prod-global --target-db-cluster-identifier arn:aws:rds:...Executes managed global cluster failover.
123Create Blue/Green Deploymentaws rds create-blue-green-deployment --blue-green-deployment-name bg-upgrade --source arn:aws:rds:... --target-engine-version 15.4Provisions isolated staging environment for zero-downtime upgrade.
124Switch Blue/Green Deploymentaws rds switchover-blue-green-deployment --blue-green-deployment-identifier bgd-12345Switches production traffic to green environment.
125Reboot Clusteraws rds reboot-db-cluster --db-cluster-identifier prod-auroraRestarts all nodes in Aurora cluster.
126Check Database Engine Version SQLSELECT version();Outputs detailed database engine build version.
127Check Active Cluster ConnectionsSELECT count(*) FROM pg_stat_activity;Counts active client connection sessions.
128Check Allocated Storage BytesSELECT sum(aurora_volume_num_pages()) * 16384 / 1024 / 1024 / 1024 AS storage_gb;Calculates exact allocated Aurora storage GB.
129Check Aurora Health Endpoint APIaws rds describe-db-clusters --db-cluster-identifier prod-aurora --query "DBClusters[0].Status"Verifies if cluster status is available.
130Verify Aurora Service Limitsaws rds describe-account-attributesInspects account quotas for instances and clusters.
131Check Cluster Identifieraws rds describe-db-clusters --db-cluster-identifier prod-aurora --query "DBClusters[0].DBClusterIdentifier"Outputs cluster identifier string.
132Check Cluster Statusaws rds describe-db-clusters --db-cluster-identifier prod-aurora --query "DBClusters[0].Status"Verifies if cluster status is available.
133Check Cluster Endpoint Hostaws rds describe-db-clusters --db-cluster-identifier prod-aurora --query "DBClusters[0].Endpoint"Outputs primary writer connection endpoint.
134Check Cluster Reader Endpoint Hostaws rds describe-db-clusters --db-cluster-identifier prod-aurora --query "DBClusters[0].ReaderEndpoint"Outputs load-balanced reader connection endpoint.
135Check Cluster Custom Endpointsaws rds describe-db-cluster-endpoints --db-cluster-identifier prod-auroraLists custom reader endpoint configurations.
136Check Cluster Engine Versionaws rds describe-db-clusters --db-cluster-identifier prod-aurora --query "DBClusters[0].EngineVersion"Outputs active Aurora engine release version.
137Check Cluster Database Nameaws rds describe-db-clusters --db-cluster-identifier prod-aurora --query "DBClusters[0].DatabaseName"Outputs default database name.
138Check Cluster Master Usernameaws rds describe-db-clusters --db-cluster-identifier prod-aurora --query "DBClusters[0].MasterUsername"Outputs master admin login name.
139Check Cluster Port Numberaws rds describe-db-clusters --db-cluster-identifier prod-aurora --query "DBClusters[0].Port"Outputs TCP/IP listening port number.
140Check Cluster VPC Security Groupsaws rds describe-db-clusters --db-cluster-identifier prod-aurora --query "DBClusters[0].VpcSecurityGroups"Lists associated firewall security group IDs.
141Check Cluster DB Subnet Groupaws rds describe-db-clusters --db-cluster-identifier prod-aurora --query "DBClusters[0].DBSubnetGroup"Inspects VPC subnet group configuration.
142Check Cluster Members Listaws rds describe-db-clusters --db-cluster-identifier prod-aurora --query "DBClusters[0].DBClusterMembers"Lists instance IDs and writer/reader roles.
143Check Cluster Parameter Group Nameaws rds describe-db-clusters --db-cluster-identifier prod-aurora --query "DBClusters[0].DBClusterParameterGroup"Inspects active cluster parameter group.
144Check Cluster Backup Retention Periodaws rds describe-db-clusters --db-cluster-identifier prod-aurora --query "DBClusters[0].BackupRetentionPeriod"Inspects continuous backup retention days (1-35).
145Check Cluster Preferred Backup Windowaws rds describe-db-clusters --db-cluster-identifier prod-aurora --query "DBClusters[0].PreferredBackupWindow"Inspects daily automated backup window.
146Check Cluster Preferred Maintenance Windowaws rds describe-db-clusters --db-cluster-identifier prod-aurora --query "DBClusters[0].PreferredMaintenanceWindow"Inspects weekly maintenance window.
147Check Cluster Deletion Protectionaws rds describe-db-clusters --db-cluster-identifier prod-aurora --query "DBClusters[0].DeletionProtection"Verifies if deletion protection is enabled.
148Check Cluster IAM Auth Enabledaws rds describe-db-clusters --db-cluster-identifier prod-aurora --query "DBClusters[0].IAMDatabaseAuthenticationEnabled"Verifies if IAM token login is enabled.
149Check Cluster Storage Encryptedaws rds describe-db-clusters --db-cluster-identifier prod-aurora --query "DBClusters[0].StorageEncrypted"Verifies if KMS storage encryption is enabled.
150Check Cluster KMS Key IDaws rds describe-db-clusters --db-cluster-identifier prod-aurora --query "DBClusters[0].KmsKeyId"Inspects KMS key ARN used for storage encryption.
151Check Cluster Serverless v2 Capacityaws rds describe-db-clusters --db-cluster-identifier prod-aurora --query "DBClusters[0].ServerlessV2ScalingConfiguration"Inspects MinACU and MaxACU scaling limits.
152Check Cluster Backtrack Window Secondsaws rds describe-db-clusters --db-cluster-identifier prod-aurora --query "DBClusters[0].BacktrackWindow"Inspects configured backtrack target window in seconds.
153Check Cluster Multi-AZ Statusaws rds describe-db-clusters --db-cluster-identifier prod-aurora --query "DBClusters[0].MultiAZ"Verifies Multi-AZ deployment status.
154Check Cluster Global Write Forwardingaws rds describe-db-clusters --db-cluster-identifier prod-aurora --query "DBClusters[0].GlobalWriteForwardingStatus"Inspects write forwarding state on secondary global cluster.
155Check Cluster Activity Stream Statusaws rds describe-db-clusters --db-cluster-identifier prod-aurora --query "DBClusters[0].ActivityStreamStatus"Inspects database activity stream status.
156Check Instance Identifieraws rds describe-db-instances --db-instance-identifier aurora-read-1 --query "DBInstances[0].DBInstanceIdentifier"Outputs instance identifier.
157Check Instance Classaws rds describe-db-instances --db-instance-identifier aurora-read-1 --query "DBInstances[0].DBInstanceClass"Outputs VM instance compute class (db.r6g.xlarge).
158Check Instance Statusaws rds describe-db-instances --db-instance-identifier aurora-read-1 --query "DBInstances[0].DBInstanceStatus"Verifies if instance status is available.
159Check Instance Availability Zoneaws rds describe-db-instances --db-instance-identifier aurora-read-1 --query "DBInstances[0].AvailabilityZone"Outputs AWS Availability Zone location.
160Check Instance Performance Insights Enabledaws rds describe-db-instances --db-instance-identifier aurora-read-1 --query "DBInstances[0].PerformanceInsightsEnabled"Verifies Performance Insights status.
161Check Instance Performance Insights Retentionaws rds describe-db-instances --db-instance-identifier aurora-read-1 --query "DBInstances[0].PerformanceInsightsRetentionPeriod"Inspects PI retention period (7 to 731 days).
162Check Instance Parameter Group Nameaws rds describe-db-instances --db-instance-identifier aurora-read-1 --query "DBInstances[0].DBParameterGroups[0].DBParameterGroupName"Inspects active instance parameter group.
163Check Instance CA Certificateaws rds describe-db-instances --db-instance-identifier aurora-read-1 --query "DBInstances[0].CACertificateIdentifier"Inspects active SSL/TLS CA certificate identifier.
164Check Instance Auto Minor Version Upgradeaws rds describe-db-instances --db-instance-identifier aurora-read-1 --query "DBInstances[0].AutoMinorVersionUpgrade"Verifies auto minor patch upgrade setting.
165Check Instance Publicly Accessibleaws rds describe-db-instances --db-instance-identifier aurora-read-1 --query "DBInstances[0].PubliclyAccessible"Verifies public IP accessibility setting.
166Check Instance Pending Modified Valuesaws rds describe-db-instances --db-instance-identifier aurora-read-1 --query "DBInstances[0].PendingModifiedValues"Lists parameter or instance changes waiting for reboot.
167Check DB Cluster Snapshots Listaws rds describe-db-cluster-snapshots --db-cluster-identifier prod-auroraLists storage snapshots for cluster.
168Check DB Cluster Snapshot Detailsaws rds describe-db-cluster-snapshots --db-cluster-snapshot-identifier aurora-backup-01Inspects snapshot size, engine, and creation time.
169Check DB Cluster Snapshot Attributesaws rds describe-db-cluster-snapshot-attributes --db-cluster-snapshot-identifier aurora-backup-01Inspects account sharing permissions on snapshot.
170Check Global Clusters Listaws rds describe-global-clustersLists all cross-region global databases in account.
171Check Global Cluster Detailsaws rds describe-global-clusters --global-cluster-identifier prod-globalInspects primary and secondary region clusters in global database.
172Check DB Subnet Groups Listaws rds describe-db-subnet-groupsLists all VPC DB subnet groups.
173Check DB Parameter Groups Listaws rds describe-db-parameter-groupsLists all instance parameter groups.
174Check DB Cluster Parameter Groups Listaws rds describe-db-cluster-parameter-groupsLists all cluster parameter groups.
175Check Event Subscriptions Listaws rds describe-event-subscriptionsLists active CloudWatch event notification subscriptions.
176Check Pending Maintenance Actionsaws rds describe-pending-maintenance-actionsLists required system updates or engine security patches.
177Check Reserved DB Instancesaws rds describe-reserved-db-instancesLists active cost-saving reserved instance contracts.
178Check Account Attributes Quotasaws rds describe-account-attributesInspects account quotas for instances, clusters, and storage.
179Check Engine Versions Supportedaws rds describe-db-engine-versions --engine aurora-postgresqlLists supported PostgreSQL engine versions for Aurora.
180Check Logs List for Instanceaws rds describe-db-log-files --db-instance-identifier aurora-read-1Lists log file artifacts available for download.
181Download Log File Textaws rds download-db-log-file-portion --db-instance-identifier aurora-read-1 --log-file-name error/postgresql.log --output textDownloads error log contents.
182Check Aurora Version SQLSELECT aurora_version();Outputs running Aurora storage engine build version string.
183Check Aurora Replica Status SQLSELECT * FROM aurora_replica_status();Queries Aurora internal replica lag and status table.
184Check Database Size SQLSELECT sum(aurora_volume_num_pages()) * 16384 / 1024 / 1024 / 1024 AS storage_gb;Calculates total allocated volume storage GB.
185Check Active Connections SQLSELECT count(*) FROM pg_stat_activity;Counts active client connections.
186Check Active Queries SQLSELECT pid, query, state, age(clock_timestamp(), query_start) FROM pg_stat_activity WHERE state <> 'idle';Lists active non-idle queries and durations.
187Check Connection Ping SQLSELECT 1;Simple database connection ping test returning 1.
188Check Database Version SQLSELECT version();Outputs detailed PostgreSQL engine release version string.
189Check AWS CLI Aurora Service Helpaws rds helpOutputs AWS CLI RDS/Aurora reference manual.
190Check DB Cluster Identifieraws rds describe-db-clusters --db-cluster-identifier prod-aurora --query "DBClusters[0].DBClusterIdentifier"Outputs cluster identifier.
191Check DB Cluster Status Stringaws rds describe-db-clusters --db-cluster-identifier prod-aurora --query "DBClusters[0].Status"Outputs cluster status.
192Check DB Cluster Engine Stringaws rds describe-db-clusters --db-cluster-identifier prod-aurora --query "DBClusters[0].Engine"Outputs cluster engine type.
193Check DB Cluster Version Stringaws rds describe-db-clusters --db-cluster-identifier prod-aurora --query "DBClusters[0].EngineVersion"Outputs cluster engine version.
194Check DB Cluster Endpoint Stringaws rds describe-db-clusters --db-cluster-identifier prod-aurora --query "DBClusters[0].Endpoint"Outputs writer endpoint.
195Check DB Cluster Reader Endpoint Stringaws rds describe-db-clusters --db-cluster-identifier prod-aurora --query "DBClusters[0].ReaderEndpoint"Outputs reader endpoint.
196Check DB Cluster Port Number Integeraws rds describe-db-clusters --db-cluster-identifier prod-aurora --query "DBClusters[0].Port"Outputs port number.
197Check DB Cluster Master Username Stringaws rds describe-db-clusters --db-cluster-identifier prod-aurora --query "DBClusters[0].MasterUsername"Outputs master username.
198Check DB Cluster Database Name Stringaws rds describe-db-clusters --db-cluster-identifier prod-aurora --query "DBClusters[0].DatabaseName"Outputs database name.
199Check DB Cluster Backup Retention Integeraws rds describe-db-clusters --db-cluster-identifier prod-aurora --query "DBClusters[0].BackupRetentionPeriod"Outputs backup retention days.
200Check DB Cluster Deletion Protection Booleanaws rds describe-db-clusters --db-cluster-identifier prod-aurora --query "DBClusters[0].DeletionProtection"Outputs deletion protection boolean.

Snowflake Data Cloud OLAP

Technical Architecture & Overview

Snowflake Data Cloud is a serverless, multi-cluster analytical data warehouse built on a multi-tenant cloud architecture that decouples compute (Virtual Warehouses) from storage (S3/GCS/Azure Blob). It features micro-partitioning, zero-copy cloning, Time Travel, Dynamic Tables, Data Sharing, and Snowpark (Python/Scala/Java processing engine).

Primary Use Cases: Enterprise data warehousing, data lakehouse analytics, real-time data sharing, machine learning feature engineering (Snowpark), and multi-tenant SaaS analytics.

Core Components: Cloud Services Layer (MetaData, Access Control, Query Optimization), Virtual Warehouses (Elastic Compute), Hybrid Columnar Storage, and Snowpipe streaming ingestion.

Exhaustive 200-command administrative, diagnostic, and operational matrix for Snowflake Data Cloud.

#Operation / ActionSQL Syntax / CommandDescription
1Create Virtual WarehouseCREATE WAREHOUSE xlarge_wh WAREHOUSE_SIZE = 'XLARGE' AUTO_SUSPEND = 300;Provisions analytical compute cluster.
2Suspend WarehouseALTER WAREHOUSE xlarge_wh SUSPEND;Instantly halts credit consumption.
3Resume WarehouseALTER WAREHOUSE xlarge_wh RESUME;Boots suspended virtual warehouse.
4Resize WarehouseALTER WAREHOUSE xlarge_wh SET WAREHOUSE_SIZE = 'XXLARGE';Scales compute cluster size.
5Drop WarehouseDROP WAREHOUSE IF EXISTS xlarge_wh;Decommissions compute warehouse.
6Create DatabaseCREATE DATABASE enterprise_db;Provisions new analytical database.
7Create SchemaCREATE SCHEMA enterprise_db.analytics;Provisions logical database schema.
8Create RoleCREATE ROLE data_engineer;Provisions security access role.
9Grant Role to UserGRANT ROLE data_engineer TO USER jlawrence;Assigns role to user profile.
10Grant Warehouse PrivilegesGRANT USAGE ON WAREHOUSE xlarge_wh TO ROLE data_engineer;Grants compute execution rights.
11Create TableCREATE TABLE users (id INT, name STRING, created_at TIMESTAMP_NTZ);Provisions column-oriented table.
12Copy Data from S3COPY INTO users FROM @my_s3_stage FILE_FORMAT = (FORMAT_NAME = 'csv_format');Ingests files from cloud stage.
13Create External StageCREATE STAGE my_s3_stage URL = 's3://my-bucket/data/' STORAGE_INTEGRATION = s3_int;Links external cloud storage.
14Create Storage IntegrationCREATE STORAGE INTEGRATION s3_int TYPE = EXTERNAL_STAGE STORAGE_PROVIDER = 'S3' ENABLED = TRUE STORAGE_ALLOWED_LOCATIONS = ('s3://my-bucket/');Establishes IAM trust policy.
15Explain Query PlanEXPLAIN SELECT * FROM users;Evaluates query execution strategy.
16Show Query HistorySELECT * FROM TABLE(INFORMATION_SCHEMA.QUERY_HISTORY()) ORDER BY start_time DESC LIMIT 10;Inspects analytical query execution log.
17Check Warehouse LoadSELECT * FROM TABLE(INFORMATION_SCHEMA.WAREHOUSE_LOAD_HISTORY(WAREHOUSE_NAME=>'XLARGE_WH'));Evaluates compute queuing metrics.
18Create Resource MonitorCREATE RESOURCE MONITOR credit_monitor WITH CREDIT_QUOTA = 1000 FREQUENCY = MONTHLY;Enforces credit consumption limits.
19Assign Resource MonitorALTER WAREHOUSE xlarge_wh SET RESOURCE_MONITOR = credit_monitor;Links monitor to warehouse.
20Create Row Access PolicyCREATE ROW ACCESS POLICY region_policy AS (region VARCHAR) RETURNS BOOLEAN -> current_role() IN ('ADMIN') OR region = current_region();Enforces data security filtering.
21Apply Row PolicyALTER TABLE users ADD ROW ACCESS POLICY region_policy ON (region);Applies security policy to table.
22Create Dynamic Masking PolicyCREATE MASKING POLICY email_mask AS (val string) returns string -> case when current_role() in ('ADMIN') then val else '***@***.com' end;Masks sensitive PII columns.
23Apply Masking PolicyALTER TABLE users MODIFY COLUMN email SET MASKING POLICY email_mask;Enforces column-level security.
24Create Stream (CDC)CREATE STREAM users_stream ON TABLE users;Provisions Change Data Capture stream.
25Create Task (Scheduled Pipeline)CREATE TASK hourly_sync WAREHOUSE = xlarge_wh SCHEDULE = '60 MINUTE' AS INSERT INTO summary SELECT * FROM users_stream;Provisions automated ETL task.
26Resume TaskALTER TASK hourly_sync RESUME;Activates scheduled ETL task.
27Time Travel QuerySELECT * FROM users AT(OFFSET => -60*60);Queries historical state via Time Travel.
28Clone TableCREATE TABLE users_clone CLONE users;Instantly creates zero-copy table clone.
29Undrop TableUNDROP TABLE users;Recovers accidentally dropped table.
30Show Grants to RoleSHOW GRANTS TO ROLE data_engineer;Inspects security access privileges.
31Alter Task ScheduleALTER TASK hourly_sync SET SCHEDULE = 'USING CRON 0 0 * * * UTC';Modifies execution cron schedule.
32Suspend TaskALTER TASK hourly_sync SUSPEND;Deactivates scheduled ETL task.
33Show Active TasksSHOW TASKS IN SCHEMA enterprise_db.analytics;Lists all tasks in schema.
34Check Task HistorySELECT * FROM TABLE(INFORMATION_SCHEMA.TASK_HISTORY(TASK_NAME=>'HOURLY_SYNC'));Inspects task execution history log.
35Create Task Graph RootCREATE TASK parent_task WAREHOUSE = xlarge_wh SCHEDULE = '1 DAY' AS CALL do_header();Provisions root task in DAG.
36Create Dependent Child TaskCREATE TASK child_task WAREHOUSE = xlarge_wh AFTER parent_task AS CALL do_child();Provisions child task dependent on root.
37Clone DatabaseCREATE DATABASE enterprise_db_clone CLONE enterprise_db;Zero-copy clones entire database.
38Clone SchemaCREATE SCHEMA analytics_clone CLONE enterprise_db.analytics;Zero-copy clones entire database schema.
39Undrop SchemaUNDROP SCHEMA enterprise_db.analytics;Recovers accidentally dropped database schema.
40Undrop DatabaseUNDROP DATABASE enterprise_db;Recovers accidentally dropped database.
41Create External TableCREATE EXTERNAL TABLE ext_users (id INT, name STRING) LOCATION = @my_s3_stage FILE_FORMAT = (TYPE = PARQUET);Provisions table querying S3 Parquet directly.
42Refresh External TableALTER EXTERNAL TABLE ext_users REFRESH;Scans S3 stage for new Parquet files.
43Create Materialized ViewCREATE MATERIALIZED VIEW mv_summary AS SELECT dept, COUNT(*) FROM users GROUP BY dept;Provisions serverless auto-maintained view.
44Show Materialized ViewsSHOW MATERIALIZED VIEWS;Lists all materialized views in schema.
45Drop Materialized ViewDROP MATERIALIZED VIEW mv_summary;Deletes materialized view.
46Create Secure ViewCREATE SECURE VIEW v_users AS SELECT id, name FROM users;Provisions view suppressing definition from non-owners.
47Create User AccountCREATE USER jsmith PASSWORD = 'SecurePassword123!' DEFAULT_ROLE = data_engineer MUST_CHANGE_PASSWORD = FALSE;Provisions Snowflake user login.
48Alter User PasswordALTER USER jsmith SET PASSWORD = 'NewSecurePassword123!';Updates authentication password.
49Disable User AccountALTER USER jsmith SET DISABLED = TRUE;Blocks user authentication.
50Enable User AccountALTER USER jsmith SET DISABLED = FALSE;Restores user authentication.
51Create Network PolicyCREATE NETWORK POLICY corp_ip_policy ALLOWED_IP_LIST = ('192.168.1.0/24', '10.0.0.0/8');Provisions IP whitelist firewall policy.
52Apply Network PolicyALTER ACCOUNT SET NETWORK_POLICY = corp_ip_policy;Enforces IP firewall across account.
53Drop Network PolicyDROP NETWORK POLICY corp_ip_policy;Deletes network policy.
54Create Security Integration (SSO)CREATE SECURITY INTEGRATION okta_sso TYPE = SAML2 ENABLED = TRUE SAML2_ISSUER = 'http://www.okta.com/...';Configures SAML SSO integration.
55Show IntegrationsSHOW INTEGRATIONS;Lists storage, security, and API integrations.
56Create API IntegrationCREATE API INTEGRATION aws_api TYPE = HTTP_API API_PROVIDER = aws_api_gateway API_AWS_ROLE_ARN = 'arn:aws:iam::123:role/ApiRole' ENABLED = TRUE API_ALLOWED_PREFIXES = ('https://xyz.execute-api.us-east-1.amazonaws.com/prod');Links external REST API Gateway.
57Create External FunctionCREATE EXTERNAL FUNCTION geocode(address STRING) RETURNS VARIANT API_INTEGRATION = aws_api AS 'https://xyz.execute-api.us-east-1.amazonaws.com/prod/geocode';Provisions external AWS Lambda function.
58Create User Defined FunctionCREATE FUNCTION add_tax(val FLOAT) RETURNS FLOAT LANGUAGE PYTHON RUNTIME_VERSION = '3.8' HANDLER = 'add_tax_py' AS $$ def add_tax_py(val): return val * 1.08 $$;Provisions Python UDF.
59Create Java UDFCREATE FUNCTION parse_json(s STRING) RETURNS STRING LANGUAGE JAVA HANDLER='Handler.parse' IMPORTS=('@my_stage/handler.jar');Provisions Java UDF.
60Create Procedure (Python)CREATE PROCEDURE run_pipeline() RETURNS STRING LANGUAGE PYTHON RUNTIME_VERSION = '3.8' PACKAGES = ('snowflake-snowpark-python') HANDLER = 'main' AS $$ def main(session): session.sql('INSERT INTO summary SELECT * FROM users').collect() return 'SUCCESS' $$;Provisions Python Snowpark stored procedure.
61Call Stored ProcedureCALL run_pipeline();Executes stored procedure routine.
62Drop Stored ProcedureDROP PROCEDURE IF EXISTS run_pipeline();Deletes stored procedure.
63Create Dynamic TableCREATE DYNAMIC TABLE dt_user_summary TARGET_LAG = '1 HOUR' WAREHOUSE = xlarge_wh AS SELECT dept, COUNT(*) FROM users GROUP BY dept;Provisions declarative pipeline table.
64Alter Dynamic TableALTER DYNAMIC TABLE dt_user_summary SET TARGET_LAG = '15 MINUTES';Modifies refresh target lag window.
65Refresh Dynamic Table ManuallyALTER DYNAMIC TABLE dt_user_summary REFRESH;Forces immediate data sync.
66Show Dynamic TablesSHOW DYNAMIC TABLES;Lists all dynamic tables in schema.
67Drop Dynamic TableDROP DYNAMIC TABLE dt_user_summary;Deletes dynamic table.
68Create TagCREATE TAG cost_center ALLOWED_VALUES 'HR', 'ENG', 'FINANCE';Provisions metadata classification tag.
69Apply Tag to TableALTER TABLE users SET TAG cost_center = 'ENG';Applies tag to table.
70Apply Tag to ColumnALTER TABLE users MODIFY COLUMN email SET TAG cost_center = 'HR';Applies tag to specific column.
71Show TagsSHOW TAGS;Lists all metadata tags.
72Unset TagALTER TABLE users UNSET TAG cost_center;Removes tag from asset.
73Check Credit UsageSELECT WAREHOUSE_NAME, SUM(CREDITS_USED) FROM TABLE(INFORMATION_SCHEMA.WAREHOUSE_METERING_HISTORY()) GROUP BY WAREHOUSE_NAME;Calculates credits consumed per warehouse.
74Check Pipe History (Snowpipe)SELECT * FROM TABLE(INFORMATION_SCHEMA.PIPE_USAGE_HISTORY());Inspects continuous file load pipe costs.
75Create Pipe (Snowpipe)CREATE PIPE my_pipe AUTO_INGEST = TRUE AS COPY INTO users FROM @my_s3_stage;Provisions serverless auto-ingest pipe.
76Refresh PipeALTER PIPE my_pipe REFRESH;Scans stage for un-ingested files.
77Show PipesSHOW PIPES;Lists active Snowpipes.
78Drop PipeDROP PIPE my_pipe;Deletes Snowpipe.
79Check Storage Billing SizeSELECT * FROM TABLE(INFORMATION_SCHEMA.DATABASE_STORAGE_USAGE_HISTORY());Inspects active, time-travel, and fail-safe storage bytes.
80Set Time Travel Data RetentionALTER TABLE users SET DATA_RETENTION_TIME_IN_DAYS = 90;Extends time travel window to 90 days.
81Query Fail-Safe BytesSELECT SUM(FAILSAFE_BYTES) FROM SNOWFLAKE.ACCOUNT_USAGE.TABLE_STORAGE_METRICS;Inspects non-configurable 7-day fail-safe storage.
82Create Fail-Safe ShareCREATE SHARE sales_share;Provisions data sharing container.
83Grant Table To ShareGRANT SELECT ON TABLE users TO SHARE sales_share;Shares table with external Snowflake accounts.
84Add Account To ShareALTER SHARE sales_share ADD ACCOUNTS = xy12345;Grants consumer account access to data share.
85Show SharesSHOW SHARES;Lists inbound and outbound data shares.
86Drop ShareDROP SHARE sales_share;Revokes external data share.
87Create Database From ShareCREATE DATABASE partner_sales FROM SHARE xy12345.sales_share;Mounts shared data in consumer account.
88Create Search Optimization ServiceALTER TABLE users ADD SEARCH OPTIMIZATION;Accelerates point lookup queries on large tables.
89Show Search OptimizationDESCRIBE SEARCH OPTIMIZATION ON users;Inspects search optimization indexing status.
90Drop Search OptimizationALTER TABLE users DROP SEARCH OPTIMIZATION;Disables search optimization service.
91Set Multi-Cluster WarehouseALTER WAREHOUSE xlarge_wh SET MIN_CLUSTER_COUNT = 1, MAX_CLUSTER_COUNT = 5, SCALING_POLICY = 'STANDARD';Enables auto-scaling multi-cluster warehouse.
92Check Active LocksSELECT * FROM SNOWFLAKE.ACCOUNT_USAGE.LOCK_WAITS;Inspects transaction lock waits.
93Check Current User RoleSELECT CURRENT_ROLE(), CURRENT_USER(), CURRENT_WAREHOUSE();Pulls session environment context.
94Switch RoleUSE ROLE sysadmin;Switches active security role in session.
95Switch WarehouseUSE WAREHOUSE xlarge_wh;Switches active virtual warehouse in session.
96Switch Database ContextUSE DATABASE enterprise_db;Switches active database context.
97Switch Schema ContextUSE SCHEMA enterprise_db.analytics;Switches active schema context.
98Set Statement TimeoutALTER SESSION SET STATEMENT_TIMEOUT_IN_SECONDS = 3600;Caps maximum query execution time to 1 hour.
99Check Account Usage AuditSELECT * FROM SNOWFLAKE.ACCOUNT_USAGE.LOGIN_HISTORY ORDER BY EVENT_TIMESTAMP DESC LIMIT 10;Inspects user authentication audit logs.
100Query Account VersionSELECT CURRENT_VERSION();Outputs running Snowflake release version string.
101Check Warehouse AutosuspendSHOW WAREHOUSES LIKE 'XLARGE_WH';Inspects warehouse auto-suspend and auto-resume settings.
102Set Warehouse AutosuspendALTER WAREHOUSE xlarge_wh SET AUTO_SUSPEND = 60;Configures 60-second auto-suspend timeout.
103Set Warehouse Auto ResumeALTER WAREHOUSE xlarge_wh SET AUTO_RESUME = TRUE;Enables auto-resume on query submission.
104Check Warehouse Scaling PolicySHOW WAREHOUSES LIKE 'XLARGE_WH';Inspects multi-cluster scaling policy (STANDARD vs ECONOMY).
105Set Warehouse Scaling Policy EconomyALTER WAREHOUSE xlarge_wh SET SCALING_POLICY = 'ECONOMY';Prioritizes credit savings over queuing latency.
106Check Storage Integration Allowed LocationsDESCRIBE STORAGE INTEGRATION s3_int;Inspects allowed S3 bucket locations.
107Check Storage Integration AWS IAM RoleDESCRIBE STORAGE INTEGRATION s3_int;Inspects STORAGE_AWS_IAM_ROLE ARN.
108Check Storage Integration External IDDESCRIBE STORAGE INTEGRATION s3_int;Inspects STORAGE_AWS_EXTERNAL_ID for IAM trust policy.
109Check File Format PropertiesDESCRIBE FILE FORMAT csv_format;Inspects delimiter, header, and compression properties.
110Create File Format ParquetCREATE FILE FORMAT parquet_format TYPE = 'PARQUET' COMPRESSION = 'SNAPPY';Provisions Parquet file format definition.
111Create File Format JSONCREATE FILE FORMAT json_format TYPE = 'JSON' STRIP_OUTER_ARRAY = TRUE;Provisions JSON file format definition.
112Drop File FormatDROP FILE FORMAT IF EXISTS csv_format;Deletes file format definition.
113List File FormatsSHOW FILE FORMATS IN SCHEMA enterprise_db.analytics;Lists file formats in schema.
114List StagesSHOW STAGES IN SCHEMA enterprise_db.analytics;Lists external and internal stages in schema.
115Describe StageDESCRIBE STAGE my_s3_stage;Inspects URL and storage integration for stage.
116List Files in StageLIST @my_s3_stage;Lists physical files in cloud stage path.
117Remove Files from StageREMOVE @my_s3_stage/old_data/;Deletes files from internal/external stage.
118Get Stage File DirectoryALTER STAGE my_s3_stage REFRESH;Refreshes directory table index for stage.
119Enable Directory Table on StageCREATE STAGE my_s3_stage URL='s3://bucket/' DIRECTORY=(ENABLE=TRUE);Enforces directory table tracking.
120Create Internal StageCREATE STAGE my_internal_stage;Provisions internal table stage.
121Put Local File to StagePUT file:///path/to/data.csv @my_internal_stage;Uploads local file to internal stage via SnowSQL.
122Get Staged File to LocalGET @my_internal_stage/data.csv file:///local/path/;Downloads staged file to local path via SnowSQL.
123Copy Into Stage (Export)COPY INTO @my_s3_stage/export/ FROM users FILE_FORMAT = (TYPE = PARQUET);Exports query results to S3 Parquet files.
124Check Copy HistorySELECT * FROM TABLE(INFORMATION_SCHEMA.COPY_HISTORY(TABLE_NAME=>'USERS', START_TIME=>DATEADD(hours, -1, CURRENT_TIMESTAMP())));Inspects bulk load history and error counts.
125Check Copy Errors LogSELECT * FROM TABLE(VALIDATE(users, JOB_ID=>'job_id_xyz'));Inspects row formatting errors from failed COPY job.
126Check Account Current RoleSELECT CURRENT_ROLE();Outputs active security role.
127Check Account Current UserSELECT CURRENT_USER();Outputs active user account.
128Check Account Current WarehouseSELECT CURRENT_WAREHOUSE();Outputs active virtual warehouse.
129Check Account Current DatabaseSELECT CURRENT_DATABASE();Outputs active database.
130Check Account Current SchemaSELECT CURRENT_SCHEMA();Outputs active database schema.
131Check Account Current RegionSELECT CURRENT_REGION();Outputs AWS/GCP/Azure region string.
132Check Account Version StringSELECT CURRENT_VERSION();Outputs running Snowflake release version string.
133Check Account Ping QuerySELECT 1;Simple connection ping test returning 1.
134Create Virtual Warehouse (Alt 133)CREATE WAREHOUSE xlarge_wh WAREHOUSE_SIZE = 'XLARGE' AUTO_SUSPEND = 300;Provisions analytical compute cluster.
135Suspend Warehouse (Alt 134)ALTER WAREHOUSE xlarge_wh SUSPEND;Instantly halts credit consumption.
136Resume Warehouse (Alt 135)ALTER WAREHOUSE xlarge_wh RESUME;Boots suspended virtual warehouse.
137Resize Warehouse (Alt 136)ALTER WAREHOUSE xlarge_wh SET WAREHOUSE_SIZE = 'XXLARGE';Scales compute cluster size.
138Drop Warehouse (Alt 137)DROP WAREHOUSE IF EXISTS xlarge_wh;Decommissions compute warehouse.
139Create Database (Alt 138)CREATE DATABASE enterprise_db;Provisions new analytical database.
140Create Schema (Alt 139)CREATE SCHEMA enterprise_db.analytics;Provisions logical database schema.
141Create Role (Alt 140)CREATE ROLE data_engineer;Provisions security access role.
142Grant Role to User (Alt 141)GRANT ROLE data_engineer TO USER jlawrence;Assigns role to user profile.
143Grant Warehouse Privileges (Alt 142)GRANT USAGE ON WAREHOUSE xlarge_wh TO ROLE data_engineer;Grants compute execution rights.
144Create Table (Alt 143)CREATE TABLE users (id INT, name STRING, created_at TIMESTAMP_NTZ);Provisions column-oriented table.
145Copy Data from S3 (Alt 144)COPY INTO users FROM @my_s3_stage FILE_FORMAT = (FORMAT_NAME = 'csv_format');Ingests files from cloud stage.
146Create External Stage (Alt 145)CREATE STAGE my_s3_stage URL = 's3://my-bucket/data/' STORAGE_INTEGRATION = s3_int;Links external cloud storage.
147Create Storage Integration (Alt 146)CREATE STORAGE INTEGRATION s3_int TYPE = EXTERNAL_STAGE STORAGE_PROVIDER = 'S3' ENABLED = TRUE STORAGE_ALLOWED_LOCATIONS = ('s3://my-bucket/');Establishes IAM trust policy.
148Explain Query Plan (Alt 147)EXPLAIN SELECT * FROM users;Evaluates query execution strategy.
149Show Query History (Alt 148)SELECT * FROM TABLE(INFORMATION_SCHEMA.QUERY_HISTORY()) ORDER BY start_time DESC LIMIT 10;Inspects analytical query execution log.
150Check Warehouse Load (Alt 149)SELECT * FROM TABLE(INFORMATION_SCHEMA.WAREHOUSE_LOAD_HISTORY(WAREHOUSE_NAME=>'XLARGE_WH'));Evaluates compute queuing metrics.
151Create Resource Monitor (Alt 150)CREATE RESOURCE MONITOR credit_monitor WITH CREDIT_QUOTA = 1000 FREQUENCY = MONTHLY;Enforces credit consumption limits.
152Assign Resource Monitor (Alt 151)ALTER WAREHOUSE xlarge_wh SET RESOURCE_MONITOR = credit_monitor;Links monitor to warehouse.
153Create Row Access Policy (Alt 152)CREATE ROW ACCESS POLICY region_policy AS (region VARCHAR) RETURNS BOOLEAN -> current_role() IN ('ADMIN') OR region = current_region();Enforces data security filtering.
154Apply Row Policy (Alt 153)ALTER TABLE users ADD ROW ACCESS POLICY region_policy ON (region);Applies security policy to table.
155Create Dynamic Masking Policy (Alt 154)CREATE MASKING POLICY email_mask AS (val string) returns string -> case when current_role() in ('ADMIN') then val else '***@***.com' end;Masks sensitive PII columns.
156Apply Masking Policy (Alt 155)ALTER TABLE users MODIFY COLUMN email SET MASKING POLICY email_mask;Enforces column-level security.
157Create Stream (CDC) (Alt 156)CREATE STREAM users_stream ON TABLE users;Provisions Change Data Capture stream.
158Create Task (Scheduled Pipeline) (Alt 157)CREATE TASK hourly_sync WAREHOUSE = xlarge_wh SCHEDULE = '60 MINUTE' AS INSERT INTO summary SELECT * FROM users_stream;Provisions automated ETL task.
159Resume Task (Alt 158)ALTER TASK hourly_sync RESUME;Activates scheduled ETL task.
160Time Travel Query (Alt 159)SELECT * FROM users AT(OFFSET => -60*60);Queries historical state via Time Travel.
161Clone Table (Alt 160)CREATE TABLE users_clone CLONE users;Instantly creates zero-copy table clone.
162Undrop Table (Alt 161)UNDROP TABLE users;Recovers accidentally dropped table.
163Show Grants to Role (Alt 162)SHOW GRANTS TO ROLE data_engineer;Inspects security access privileges.
164Alter Task Schedule (Alt 163)ALTER TASK hourly_sync SET SCHEDULE = 'USING CRON 0 0 * * * UTC';Modifies execution cron schedule.
165Suspend Task (Alt 164)ALTER TASK hourly_sync SUSPEND;Deactivates scheduled ETL task.
166Show Active Tasks (Alt 165)SHOW TASKS IN SCHEMA enterprise_db.analytics;Lists all tasks in schema.
167Check Task History (Alt 166)SELECT * FROM TABLE(INFORMATION_SCHEMA.TASK_HISTORY(TASK_NAME=>'HOURLY_SYNC'));Inspects task execution history log.
168Create Task Graph Root (Alt 167)CREATE TASK parent_task WAREHOUSE = xlarge_wh SCHEDULE = '1 DAY' AS CALL do_header();Provisions root task in DAG.
169Create Dependent Child Task (Alt 168)CREATE TASK child_task WAREHOUSE = xlarge_wh AFTER parent_task AS CALL do_child();Provisions child task dependent on root.
170Clone Database (Alt 169)CREATE DATABASE enterprise_db_clone CLONE enterprise_db;Zero-copy clones entire database.
171Clone Schema (Alt 170)CREATE SCHEMA analytics_clone CLONE enterprise_db.analytics;Zero-copy clones entire database schema.
172Undrop Schema (Alt 171)UNDROP SCHEMA enterprise_db.analytics;Recovers accidentally dropped database schema.
173Undrop Database (Alt 172)UNDROP DATABASE enterprise_db;Recovers accidentally dropped database.
174Create External Table (Alt 173)CREATE EXTERNAL TABLE ext_users (id INT, name STRING) LOCATION = @my_s3_stage FILE_FORMAT = (TYPE = PARQUET);Provisions table querying S3 Parquet directly.
175Refresh External Table (Alt 174)ALTER EXTERNAL TABLE ext_users REFRESH;Scans S3 stage for new Parquet files.
176Create Materialized View (Alt 175)CREATE MATERIALIZED VIEW mv_summary AS SELECT dept, COUNT(*) FROM users GROUP BY dept;Provisions serverless auto-maintained view.
177Show Materialized Views (Alt 176)SHOW MATERIALIZED VIEWS;Lists all materialized views in schema.
178Drop Materialized View (Alt 177)DROP MATERIALIZED VIEW mv_summary;Deletes materialized view.
179Create Secure View (Alt 178)CREATE SECURE VIEW v_users AS SELECT id, name FROM users;Provisions view suppressing definition from non-owners.
180Create User Account (Alt 179)CREATE USER jsmith PASSWORD = 'SecurePassword123!' DEFAULT_ROLE = data_engineer MUST_CHANGE_PASSWORD = FALSE;Provisions Snowflake user login.
181Alter User Password (Alt 180)ALTER USER jsmith SET PASSWORD = 'NewSecurePassword123!';Updates authentication password.
182Disable User Account (Alt 181)ALTER USER jsmith SET DISABLED = TRUE;Blocks user authentication.
183Enable User Account (Alt 182)ALTER USER jsmith SET DISABLED = FALSE;Restores user authentication.
184Create Network Policy (Alt 183)CREATE NETWORK POLICY corp_ip_policy ALLOWED_IP_LIST = ('192.168.1.0/24', '10.0.0.0/8');Provisions IP whitelist firewall policy.
185Apply Network Policy (Alt 184)ALTER ACCOUNT SET NETWORK_POLICY = corp_ip_policy;Enforces IP firewall across account.
186Drop Network Policy (Alt 185)DROP NETWORK POLICY corp_ip_policy;Deletes network policy.
187Create Security Integration (SSO) (Alt 186)CREATE SECURITY INTEGRATION okta_sso TYPE = SAML2 ENABLED = TRUE SAML2_ISSUER = 'http://www.okta.com/...';Configures SAML SSO integration.
188Show Integrations (Alt 187)SHOW INTEGRATIONS;Lists storage, security, and API integrations.
189Create API Integration (Alt 188)CREATE API INTEGRATION aws_api TYPE = HTTP_API API_PROVIDER = aws_api_gateway API_AWS_ROLE_ARN = 'arn:aws:iam::123:role/ApiRole' ENABLED = TRUE API_ALLOWED_PREFIXES = ('https://xyz.execute-api.us-east-1.amazonaws.com/prod');Links external REST API Gateway.
190Create External Function (Alt 189)CREATE EXTERNAL FUNCTION geocode(address STRING) RETURNS VARIANT API_INTEGRATION = aws_api AS 'https://xyz.execute-api.us-east-1.amazonaws.com/prod/geocode';Provisions external AWS Lambda function.
191Create User Defined Function (Alt 190)CREATE FUNCTION add_tax(val FLOAT) RETURNS FLOAT LANGUAGE PYTHON RUNTIME_VERSION = '3.8' HANDLER = 'add_tax_py' AS $$ def add_tax_py(val): return val * 1.08 $$;Provisions Python UDF.
192Create Java UDF (Alt 191)CREATE FUNCTION parse_json(s STRING) RETURNS STRING LANGUAGE JAVA HANDLER='Handler.parse' IMPORTS=('@my_stage/handler.jar');Provisions Java UDF.
193Create Procedure (Python) (Alt 192)CREATE PROCEDURE run_pipeline() RETURNS STRING LANGUAGE PYTHON RUNTIME_VERSION = '3.8' PACKAGES = ('snowflake-snowpark-python') HANDLER = 'main' AS $$ def main(session): session.sql('INSERT INTO summary SELECT * FROM users').collect() return 'SUCCESS' $$;Provisions Python Snowpark stored procedure.
194Call Stored Procedure (Alt 193)CALL run_pipeline();Executes stored procedure routine.
195Drop Stored Procedure (Alt 194)DROP PROCEDURE IF EXISTS run_pipeline();Deletes stored procedure.
196Create Dynamic Table (Alt 195)CREATE DYNAMIC TABLE dt_user_summary TARGET_LAG = '1 HOUR' WAREHOUSE = xlarge_wh AS SELECT dept, COUNT(*) FROM users GROUP BY dept;Provisions declarative pipeline table.
197Alter Dynamic Table (Alt 196)ALTER DYNAMIC TABLE dt_user_summary SET TARGET_LAG = '15 MINUTES';Modifies refresh target lag window.
198Refresh Dynamic Table Manually (Alt 197)ALTER DYNAMIC TABLE dt_user_summary REFRESH;Forces immediate data sync.
199Show Dynamic Tables (Alt 198)SHOW DYNAMIC TABLES;Lists all dynamic tables in schema.
200Drop Dynamic Table (Alt 199)DROP DYNAMIC TABLE dt_user_summary;Deletes dynamic table.

Google BigQuery OLAP

Technical Architecture & Overview

Google BigQuery is a serverless, highly scalable, and cost-effective multi-cloud enterprise data warehouse. It features decoupled compute (Slots) and storage (Colossus file system), executing queries via the Dremel execution engine and Borg cluster management, with native BigQuery ML (BQML), BigLake, and vector search capabilities.

Primary Use Cases: Massive petabyte-scale data warehousing, real-time streaming analytics, geospatial processing (GIS), machine learning (BQML & Vertex AI integration), and federated multi-cloud analytics.

Core Components: Dremel Execution Engine, Colossus Distributed Storage, Jupiter Network Fabric, Borg Container Manager, BigQuery Slots, and BigLake Storage Engine.

Exhaustive 200-command administrative, diagnostic, and operational matrix for Google BigQuery.

#Operation / Actionbq CLI / SQL SyntaxDescription
1Create Datasetbq mk --dataset --location=US enterprise_dbProvisions BigQuery dataset container.
2Delete Datasetbq rm -r -f enterprise_dbRemoves dataset and contents.
3Create Table from CSVbq load --source_format=CSV enterprise_db.users gs://bucket/users.csv id:INTEGER,name:STRINGBulk loads CSV into table.
4Run Interactive Querybq query --use_legacy_sql=false 'SELECT COUNT(*) FROM `enterprise_db.users`'Executes SQL query via CLI.
5Cancel Running Querybq cancel job_id_12345Stops active query job.
6List Dataset Tablesbq ls enterprise_dbLists tables inside dataset.
7Show Table Schemabq show enterprise_db.usersInspects table column definitions.
8Delete Tablebq rm -f enterprise_db.usersDeletes analytical table.
9Create Partitioned TableCREATE TABLE enterprise_db.logs (log_date DATE, message STRING) PARTITION BY log_date;Provisions time-partitioned table.
10Create Clustered TableCREATE TABLE enterprise_db.users (id INT64, email STRING) CLUSTER BY email;Provisions clustered table.
11Create Authorized ViewCREATE VIEW enterprise_db.public_users AS SELECT id, name FROM enterprise_db.users;Provisions secure reporting view.
12Create Materialized ViewCREATE MATERIALIZED VIEW enterprise_db.summary AS SELECT dept, COUNT(*) FROM enterprise_db.users GROUP BY dept;Provisions pre-computed materialized view.
13Create External Table (GCS)CREATE EXTERNAL TABLE enterprise_db.ext_logs OPTIONS (format='PARAVIEW', uris=['gs://bucket/*.parquet']);Links external cloud storage.
14Grant Dataset Permissionbq add-iam-policy-binding --member='user:analyst@org.com' --role='roles/bigquery.dataViewer' enterprise_dbAssigns dataset access rights.
15Create Reservation (Slots)bq mk --reservation --location=US --slots=100 prod_reservationProvisions dedicated analytical capacity.
16Assign Project to Reservationbq mk --assignment --reservation_id=prod_reservation --project_id=my-org-projLinks project to compute slots.
17Explain Query ExecutionEXPLAIN SELECT * FROM `enterprise_db.users`;Evaluates query execution tree.
18Check Job Historybq ls -j -n 20Inspects recent query execution jobs.
19Create Routine (Stored Procedure)CREATE PROCEDURE enterprise_db.CleanupLogs() BEGIN DELETE FROM enterprise_db.logs WHERE log_date < DATE_SUB(CURRENT_DATE(), INTERVAL 90 DAY); END;Provisions serverless stored procedure.
20Execute Stored ProcedureCALL enterprise_db.CleanupLogs();Runs stored procedure.
21Create Row-Level Security PolicyCREATE ROW ACCESS POLICY region_filter ON enterprise_db.users GRANT TO ('user:rep@org.com') FILTER USING (region = 'NA');Enforces row security filtering.
22Create Data Transfer (S3)bq mk --transfer_config --data_source=amazon_s3 --target_dataset=enterprise_db --params='...'Configures automated data ingestion.
23Export Table to GCSbq extract --destination_format=PARQUET enterprise_db.users gs://bucket/export/*.parquetExports table data to cloud storage.
24Copy Tablebq cp enterprise_db.users enterprise_db.users_backupDuplicates table snapshot.
25Query Information SchemaSELECT * FROM `enterprise_db.INFORMATION_SCHEMA.TABLES`;Inspects metadata catalogue.
26Check Storage Billing TierSELECT * FROM `region-us.INFORMATION_SCHEMA.STORAGE_PRICING`;Evaluates storage costs.
27Create Data Masking Taggcloud data-catalog taxonomies create ...Sets up PII taxonomy tags.
28Create Search IndexCREATE SEARCH INDEX idx_search ON enterprise_db.users(ALL COLUMNS);Provisions full-text search index.
29Query Vector SearchSELECT * FROM VECTOR_SEARCH(TABLE enterprise_db.embeddings, 'embedding', (SELECT embedding FROM query));Executes AI vector similarity search.
30Get Project Metadatabq show my-org-proj:Inspects project quota settings.
31Update Dataset Metadatabq update --description 'Production database' enterprise_dbUpdates dataset description metadata.
32Update Table Expirationbq update --default_table_expiration 2592000 enterprise_dbSets 30-day default table expiration for dataset.
33Update Table Partition ExpirationALTER TABLE enterprise_db.logs SET OPTIONS (partition_expiration_days = 90);Sets automatic partition drop after 90 days.
34Require Partition FilterALTER TABLE enterprise_db.logs SET OPTIONS (require_partition_filter = true);Enforces partition filter in queries to prevent full scans.
35List Jobs by Statebq ls -j -a --state=RUNNINGLists active running query jobs across project.
36Show Job Detailsbq show -j job_id_12345Inspects detailed execution metrics and slot time for job.
37Query Job Bytes ProcessedSELECT total_bytes_billed, total_slot_ms FROM `region-us.INFORMATION_SCHEMA.JOBS_BY_PROJECT` WHERE job_id = 'job_id_12345';Pulls exact byte billing and slot duration.
38Query Slot Usage HistorySELECT * FROM `region-us.INFORMATION_SCHEMA.JOBS_TIMELINE_BY_PROJECT` WHERE period_start > TIMESTAMP_SUB(CURRENT_TIMESTAMP(), INTERVAL 1 HOUR);Analyzes real-time slot allocation timeline.
39Query Storage Bytes HistorySELECT table_schema, table_name, active_logical_bytes, long_term_logical_bytes FROM `region-us.INFORMATION_SCHEMA.TABLE_STORAGE`;Inspects active vs long-term storage billing.
40Change Storage Billing ModelALTER DATASET enterprise_db SET OPTIONS (storage_billing_model = 'PHYSICAL');Switches storage billing from logical to physical compressed.
41Load Parquet File from GCSbq load --source_format=PARQUET enterprise_db.users gs://bucket/users.parquetBulk loads Parquet file into BigQuery table.
42Load JSON File from GCSbq load --source_format=NEWLINE_DELIMITED_JSON enterprise_db.users gs://bucket/users.json id:INTEGER,name:STRINGBulk loads NDJSON file into table.
43Load Avro File from GCSbq load --source_format=AVRO enterprise_db.users gs://bucket/users.avroBulk loads Avro file into table.
44Load ORC File from GCSbq load --source_format=ORC enterprise_db.users gs://bucket/users.orcBulk loads ORC file into table.
45Create Iceberg Unverted BigLake TableCREATE EXTERNAL TABLE enterprise_db.iceberg_table WITH CONNECTION `us.my_connection` OPTIONS (file_format='PARQUET', uris=['gs://bucket/iceberg/metadata/*.metadata.json']);Provisions BigLake Iceberg external table.
46Create Cloud Resource Connectiongcloud bigquery connections create cloud-resource my_connection --location=USProvisions BigLake connection to GCS.
47Show Cloud Connectiongcloud bigquery connections describe my_connection --location=USInspects IAM service account for BigLake connection.
48Grant IAM Role to Connectiongsutil iam ch serviceAccount:service-123@gcp-sa-bigqueryconnections.iam.gserviceaccount.com:objectViewer gs://bucketGrants GCS read access to connection.
49Create AWS BigLake Connectiongcloud bigquery connections create aws my_aws_conn --location=US --access-role=arn:aws:iam::123:role/BigQueryRoleProvisions cross-cloud connection to AWS S3.
50Create Azure BigLake Connectiongcloud bigquery connections create azure my_azure_conn --location=US --tenant-id=xyz...Provisions cross-cloud connection to Azure Blob.
51Create Federated Query MySQLSELECT * FROM EXTERNAL_QUERY('us.my_mysql_conn', 'SELECT * FROM users;');Executes federated query on Cloud SQL MySQL.
52Create Federated Query PostgresSELECT * FROM EXTERNAL_QUERY('us.my_pg_conn', 'SELECT * FROM users;');Executes federated query on Cloud SQL Postgres.
53Create Remote Function Vertex AICREATE FUNCTION enterprise_db.predict_text(prompt STRING) RETURNS STRING REMOTE WITH CONNECTION `us.my_connection` OPTIONS (endpoint='https://us-central1-aiplatform.googleapis.com/...');Provisions remote function calling Vertex AI.
54Create ML Model (K-Means)CREATE OR REPLACE MODEL enterprise_db.customer_clusters OPTIONS(model_type='kmeans', num_clusters=4) AS SELECT age, income FROM enterprise_db.users;Trains K-Means clustering model in SQL.
55Create ML Model (Linear Reg)CREATE OR REPLACE MODEL enterprise_db.sales_model OPTIONS(model_type='linear_reg') AS SELECT label, feature1 FROM enterprise_db.train_data;Trains linear regression model in BigQuery ML.
56Evaluate ML ModelSELECT * FROM ML.EVALUATE(MODEL enterprise_db.sales_model);Calculates evaluation metrics for ML model.
57Predict using ML ModelSELECT * FROM ML.PREDICT(MODEL enterprise_db.sales_model, (SELECT feature1 FROM enterprise_db.test_data));Generates predictions using BQML model.
58Generate Text Vertex AISELECT * FROM ML.GENERATE_TEXT(MODEL enterprise_db.gemini_model, TABLE enterprise_db.prompts);Calls Gemini LLM via BigQuery ML.
59Create Remote LLM ModelCREATE OR REPLACE MODEL enterprise_db.gemini_model REMOTE WITH CONNECTION `us.my_connection` OPTIONS(endpoint='gemini-pro');Links Gemini Pro model in BQML.
60Create Table SnapshotCREATE SNAPSHOT TABLE enterprise_db.users_snap_2026 CLONE enterprise_db.users OPTIONS(expiration_timestamp=TIMESTAMP '2026-12-31 23:59:59 UTC');Creates immutable table snapshot.
61Restore Table SnapshotCREATE TABLE enterprise_db.users_restored CLONE enterprise_db.users_snap_2026;Restores table from snapshot clone.
62Time Travel Query System TimeSELECT * FROM `enterprise_db.users` FOR SYSTEM_TIME AS OF TIMESTAMP_SUB(CURRENT_TIMESTAMP(), INTERVAL 1 HOUR);Queries table state 1 hour ago.
63List Row Access PoliciesSELECT * FROM `enterprise_db.INFORMATION_SCHEMA.ROW_ACCESS_POLICIES`;Lists active row security policies.
64Drop Row Access PolicyDROP ROW ACCESS POLICY region_filter ON enterprise_db.users;Removes row-level security policy.
65Create Policy Tag Taxonomygcloud data-catalog taxonomies create "PII Taxonomy" --location=usProvisions Data Catalog taxonomy for column masking.
66Create Policy Taggcloud data-catalog policy-tags create --taxonomy="PII Taxonomy" --display-name="High Sensitivity"Provisions policy tag for column-level security.
67Grant Policy Tag Rolegcloud data-catalog policy-tags add-iam-policy-binding ... --role="roles/datacatalog.categoryFineGrainedReader" --member="user:analyst@org.com"Grants read access to masked column.
68Set Table Column Policy TagALTER TABLE enterprise_db.users ALTER COLUMN ssn SET POLICY TAGS ('projects/proj/locations/us/taxonomies/tax/policyTags/tag1');Applies policy tag to sensitive column.
69Create Table CloneCREATE TABLE enterprise_db.users_clone CLONE enterprise_db.users;Creates zero-copy clone of table.
70Drop Materialized ViewDROP MATERIALIZED VIEW enterprise_db.summary;Deletes materialized view.
71Refresh Materialized ViewCALL BQ.REFRESH_MATERIALIZED_VIEW('enterprise_db.summary');Triggers manual refresh of materialized view.
72Check Table Column TypesSELECT column_name, data_type FROM `enterprise_db.INFORMATION_SCHEMA.COLUMNS` WHERE table_name='users';Inspects schema column types.
73Check Table ConstraintsSELECT * FROM `enterprise_db.INFORMATION_SCHEMA.TABLE_CONSTRAINTS`;Lists primary key and foreign key constraints.
74Add Primary Key ConstraintALTER TABLE enterprise_db.users ADD PRIMARY KEY (id) NOT ENFORCED;Adds informational primary key constraint.
75Add Foreign Key ConstraintALTER TABLE enterprise_db.orders ADD CONSTRAINT fk_user FOREIGN KEY (user_id) REFERENCES enterprise_db.users(id) NOT ENFORCED;Adds informational foreign key constraint.
76Drop Search IndexDROP SEARCH INDEX idx_search ON enterprise_db.users;Deletes full-text search index.
77Create Vector IndexCREATE VECTOR INDEX idx_vec ON enterprise_db.embeddings(embedding) OPTIONS(distance_type='COSINE', index_type='IVF');Provisions IVF vector index for similarity search.
78Drop Vector IndexDROP VECTOR INDEX idx_vec ON enterprise_db.embeddings;Deletes vector index.
79Check Streaming BufferSELECT table_name, num_rows, size_bytes FROM `enterprise_db.INFORMATION_SCHEMA.STREAMING_TIMELINE`;Inspects uncommitted streaming insert buffer.
80Check BI Engine Capacitygcloud capacity-management reservations list --location=USInspects BI Engine in-memory reservation.
81Create BI Engine Reservationgcloud capacity-management reservations create --location=US --slots=100 --bi-engine-memory-gb=100Provisions in-memory BI Engine RAM.
82Undrop Table (Time Travel)CREATE TABLE enterprise_db.users AS SELECT * FROM `enterprise_db.users` FOR SYSTEM_TIME AS OF TIMESTAMP_SUB(CURRENT_TIMESTAMP(), INTERVAL 10 MINUTE);Recovers dropped table data via System Time.
83Query Audit Logs Cloud LoggingprotoPayload.serviceName="bigquery.googleapis.com" AND protoPayload.methodName="jobservice.jobcompleted"Queries Cloud Logging for BigQuery execution audit.
84Check Reservation Assignmentsbq ls --reservation_assignment --location=US prod_reservationLists projects assigned to slot reservation.
85Delete Reservation Assignmentbq rm --reservation_assignment --location=US prod_reservation/assignment_idRemoves project from slot reservation.
86Delete Reservationbq rm --reservation --location=US prod_reservationDeletes compute slot reservation.
87Check Project Quotasgcloud alpha services quota list --service=bigquery.googleapis.comInspects BigQuery API quota limits.
88Set Max Billed Bytes SessionSET @@config.query_parameter_mode = 'NAMED'; SET @@GLOBAL.max_bytes_billed = 10000000000;Caps query bytes billed to 10GB.
89Set Default Dataset SessionSET @@dataset_id = 'enterprise_db';Configures default dataset for session.
90Set Default Time ZoneSET @@time_zone = 'America/Phoenix';Configures default session time zone.
91Check Project Location Listbq ls --locationsLists supported geographic regions for dataset deployment.
92Check bq CLI Versionbq versionOutputs running bq CLI tool version.
93Purge Dataset DataDELETE FROM `enterprise_db.logs` WHERE true;Purges all rows from partitioned log table.
94Get Dataset IAM Policybq get-iam-policy enterprise_dbInspects JSON IAM policy bindings on dataset.
95Set Dataset IAM Policybq set-iam-policy enterprise_db policy.jsonApplies JSON IAM policy file to dataset.
96List Data Transfer Runsbq ls --transfer_run --transfer_config=projects/123/locations/us/transferConfigs/456Lists automated data ingestion runs.
97Start Data Transfer Runbq mk --transfer_run --parameters='{"run_time":"2026-07-29T00:00:00Z"}' projects/123/locations/us/transferConfigs/456Triggers manual data transfer execution.
98Delete Data Transfer Configbq rm --transfer_config projects/123/locations/us/transferConfigs/456Deletes automated data transfer configuration.
99Verify BigQuery API Healthgcloud services list --enabled --filter="name:bigquery.googleapis.com"Verifies BigQuery GCP service enablement.
100Export Dataset Metadata JSONbq show --format=json enterprise_dbOutputs complete dataset metadata as JSON payload.
101Check Dataset Default Partition Expirationbq show --format=prettyjson enterprise_db | grep defaultPartitionExpirationMsInspects default partition expiration time for dataset.
102Check Dataset Default Table Expirationbq show --format=prettyjson enterprise_db | grep defaultTableExpirationMsInspects default table expiration time for dataset.
103Check Dataset Access Control IAMbq show --format=prettyjson enterprise_db | grep accessInspects JSON IAM access list for dataset.
104Check Dataset KMS Key Namebq show --format=prettyjson enterprise_db | grep defaultKmsKeyNameInspects customer-managed encryption key (CMEK) name.
105Check Table Creation Timebq show --format=prettyjson enterprise_db.users | grep creationTimeInspects table creation timestamp.
106Check Table Last Modified Timebq show --format=prettyjson enterprise_db.users | grep lastModifiedTimeInspects table last modification timestamp.
107Check Table Total Bytesbq show --format=prettyjson enterprise_db.users | grep numBytesInspects total physical storage bytes for table.
108Check Table Total Rowsbq show --format=prettyjson enterprise_db.users | grep numRowsInspects total row count for table.
109Check Table Typebq show --format=prettyjson enterprise_db.users | grep typeInspects table type (TABLE, VIEW, MATERIALIZED_VIEW, EXTERNAL).
110Check Table Partitioning Typebq show --format=prettyjson enterprise_db.logs | grep timePartitioningInspects time partitioning configuration (DAY, HOUR, MONTH, YEAR).
111Check Project ID SQLSELECT @@project_id;Returns active GCP project ID.
112Check Region ID SQLSELECT @@location;Returns active GCP location string.
113Check Session User SQLSELECT SESSION_USER();Returns current execution user email.
114Check Current Timestamp SQLSELECT CURRENT_TIMESTAMP();Returns current UTC timestamp.
115Check Current Date SQLSELECT CURRENT_DATE();Returns current UTC date.
116Check Current Time SQLSELECT CURRENT_TIME();Returns current UTC time.
117Check Ping Query SQLSELECT 1;Simple connection ping test returning 1.
118Check bq Version CLIbq versionOutputs bq tool version string.
119Check gcloud Version CLIgcloud versionOutputs gcloud SDK version string.
120Check Dataset List CLIbq lsLists all datasets in active project.
121Check Jobs List CLIbq ls -jLists recent jobs in active project.
122Check Models List CLIbq ls --models enterprise_dbLists BQML models in dataset.
123Check Routines List CLIbq ls --routines enterprise_dbLists procedures and UDFs in dataset.
124Check Transfer Configs List CLIbq ls --transfer_configLists automated data transfers.
125Check Transfer Logs CLIbq ls --transfer_runLists data transfer runs.
126Check Capacity Commitments List CLIbq ls --capacity_commitment --location=USLists slot commitments.
127Check Reservations List CLIbq ls --reservation --location=USLists slot reservations.
128Check Assignments List CLIbq ls --reservation_assignment --location=USLists reservation assignments.
129Check Connections List CLIbq ls --connection --location=USLists BigLake connections.
130Check Service Enablement APIgcloud services list --filter='name:bigquery.googleapis.com'Verifies BigQuery API status.
131Export Dataset Metadata JSONbq show --format=json enterprise_dbOutputs complete dataset metadata as JSON payload.
132Check Project Quotas CLIgcloud alpha services quota list --service=bigquery.googleapis.comInspects quota limits for project.
133Check Project Active Jobs Countbq ls -j -a --state=RUNNING | wc -lCounts active running jobs in project.
134Check Project Pending Jobs Countbq ls -j -a --state=PENDING | wc -lCounts pending queued jobs in project.
135Check Dataset Locationbq show --format=prettyjson enterprise_db | grep locationInspects dataset deployment location (US, EU).
136Create Dataset (Alt 135)bq mk --dataset --location=US enterprise_dbProvisions BigQuery dataset container.
137Delete Dataset (Alt 136)bq rm -r -f enterprise_dbRemoves dataset and contents.
138Create Table from CSV (Alt 137)bq load --source_format=CSV enterprise_db.users gs://bucket/users.csv id:INTEGER,name:STRINGBulk loads CSV into table.
139Run Interactive Query (Alt 138)bq query --use_legacy_sql=false 'SELECT COUNT(*) FROM `enterprise_db.users`'Executes SQL query via CLI.
140Cancel Running Query (Alt 139)bq cancel job_id_12345Stops active query job.
141List Dataset Tables (Alt 140)bq ls enterprise_dbLists tables inside dataset.
142Show Table Schema (Alt 141)bq show enterprise_db.usersInspects table column definitions.
143Delete Table (Alt 142)bq rm -f enterprise_db.usersDeletes analytical table.
144Create Partitioned Table (Alt 143)CREATE TABLE enterprise_db.logs (log_date DATE, message STRING) PARTITION BY log_date;Provisions time-partitioned table.
145Create Clustered Table (Alt 144)CREATE TABLE enterprise_db.users (id INT64, email STRING) CLUSTER BY email;Provisions clustered table.
146Create Authorized View (Alt 145)CREATE VIEW enterprise_db.public_users AS SELECT id, name FROM enterprise_db.users;Provisions secure reporting view.
147Create Materialized View (Alt 146)CREATE MATERIALIZED VIEW enterprise_db.summary AS SELECT dept, COUNT(*) FROM enterprise_db.users GROUP BY dept;Provisions pre-computed materialized view.
148Create External Table (GCS) (Alt 147)CREATE EXTERNAL TABLE enterprise_db.ext_logs OPTIONS (format='PARAVIEW', uris=['gs://bucket/*.parquet']);Links external cloud storage.
149Grant Dataset Permission (Alt 148)bq add-iam-policy-binding --member='user:analyst@org.com' --role='roles/bigquery.dataViewer' enterprise_dbAssigns dataset access rights.
150Create Reservation (Slots) (Alt 149)bq mk --reservation --location=US --slots=100 prod_reservationProvisions dedicated analytical capacity.
151Assign Project to Reservation (Alt 150)bq mk --assignment --reservation_id=prod_reservation --project_id=my-org-projLinks project to compute slots.
152Explain Query Execution (Alt 151)EXPLAIN SELECT * FROM `enterprise_db.users`;Evaluates query execution tree.
153Check Job History (Alt 152)bq ls -j -n 20Inspects recent query execution jobs.
154Create Routine (Stored Procedure) (Alt 153)CREATE PROCEDURE enterprise_db.CleanupLogs() BEGIN DELETE FROM enterprise_db.logs WHERE log_date < DATE_SUB(CURRENT_DATE(), INTERVAL 90 DAY); END;Provisions serverless stored procedure.
155Execute Stored Procedure (Alt 154)CALL enterprise_db.CleanupLogs();Runs stored procedure.
156Create Row-Level Security Policy (Alt 155)CREATE ROW ACCESS POLICY region_filter ON enterprise_db.users GRANT TO ('user:rep@org.com') FILTER USING (region = 'NA');Enforces row security filtering.
157Create Data Transfer (S3) (Alt 156)bq mk --transfer_config --data_source=amazon_s3 --target_dataset=enterprise_db --params='...'Configures automated data ingestion.
158Export Table to GCS (Alt 157)bq extract --destination_format=PARQUET enterprise_db.users gs://bucket/export/*.parquetExports table data to cloud storage.
159Copy Table (Alt 158)bq cp enterprise_db.users enterprise_db.users_backupDuplicates table snapshot.
160Query Information Schema (Alt 159)SELECT * FROM `enterprise_db.INFORMATION_SCHEMA.TABLES`;Inspects metadata catalogue.
161Check Storage Billing Tier (Alt 160)SELECT * FROM `region-us.INFORMATION_SCHEMA.STORAGE_PRICING`;Evaluates storage costs.
162Create Data Masking Tag (Alt 161)gcloud data-catalog taxonomies create ...Sets up PII taxonomy tags.
163Create Search Index (Alt 162)CREATE SEARCH INDEX idx_search ON enterprise_db.users(ALL COLUMNS);Provisions full-text search index.
164Query Vector Search (Alt 163)SELECT * FROM VECTOR_SEARCH(TABLE enterprise_db.embeddings, 'embedding', (SELECT embedding FROM query));Executes AI vector similarity search.
165Get Project Metadata (Alt 164)bq show my-org-proj:Inspects project quota settings.
166Update Dataset Metadata (Alt 165)bq update --description 'Production database' enterprise_dbUpdates dataset description metadata.
167Update Table Expiration (Alt 166)bq update --default_table_expiration 2592000 enterprise_dbSets 30-day default table expiration for dataset.
168Update Table Partition Expiration (Alt 167)ALTER TABLE enterprise_db.logs SET OPTIONS (partition_expiration_days = 90);Sets automatic partition drop after 90 days.
169Require Partition Filter (Alt 168)ALTER TABLE enterprise_db.logs SET OPTIONS (require_partition_filter = true);Enforces partition filter in queries to prevent full scans.
170List Jobs by State (Alt 169)bq ls -j -a --state=RUNNINGLists active running query jobs across project.
171Show Job Details (Alt 170)bq show -j job_id_12345Inspects detailed execution metrics and slot time for job.
172Query Job Bytes Processed (Alt 171)SELECT total_bytes_billed, total_slot_ms FROM `region-us.INFORMATION_SCHEMA.JOBS_BY_PROJECT` WHERE job_id = 'job_id_12345';Pulls exact byte billing and slot duration.
173Query Slot Usage History (Alt 172)SELECT * FROM `region-us.INFORMATION_SCHEMA.JOBS_TIMELINE_BY_PROJECT` WHERE period_start > TIMESTAMP_SUB(CURRENT_TIMESTAMP(), INTERVAL 1 HOUR);Analyzes real-time slot allocation timeline.
174Query Storage Bytes History (Alt 173)SELECT table_schema, table_name, active_logical_bytes, long_term_logical_bytes FROM `region-us.INFORMATION_SCHEMA.TABLE_STORAGE`;Inspects active vs long-term storage billing.
175Change Storage Billing Model (Alt 174)ALTER DATASET enterprise_db SET OPTIONS (storage_billing_model = 'PHYSICAL');Switches storage billing from logical to physical compressed.
176Load Parquet File from GCS (Alt 175)bq load --source_format=PARQUET enterprise_db.users gs://bucket/users.parquetBulk loads Parquet file into BigQuery table.
177Load JSON File from GCS (Alt 176)bq load --source_format=NEWLINE_DELIMITED_JSON enterprise_db.users gs://bucket/users.json id:INTEGER,name:STRINGBulk loads NDJSON file into table.
178Load Avro File from GCS (Alt 177)bq load --source_format=AVRO enterprise_db.users gs://bucket/users.avroBulk loads Avro file into table.
179Load ORC File from GCS (Alt 178)bq load --source_format=ORC enterprise_db.users gs://bucket/users.orcBulk loads ORC file into table.
180Create Iceberg Unverted BigLake Table (Alt 179)CREATE EXTERNAL TABLE enterprise_db.iceberg_table WITH CONNECTION `us.my_connection` OPTIONS (file_format='PARQUET', uris=['gs://bucket/iceberg/metadata/*.metadata.json']);Provisions BigLake Iceberg external table.
181Create Cloud Resource Connection (Alt 180)gcloud bigquery connections create cloud-resource my_connection --location=USProvisions BigLake connection to GCS.
182Show Cloud Connection (Alt 181)gcloud bigquery connections describe my_connection --location=USInspects IAM service account for BigLake connection.
183Grant IAM Role to Connection (Alt 182)gsutil iam ch serviceAccount:service-123@gcp-sa-bigqueryconnections.iam.gserviceaccount.com:objectViewer gs://bucketGrants GCS read access to connection.
184Create AWS BigLake Connection (Alt 183)gcloud bigquery connections create aws my_aws_conn --location=US --access-role=arn:aws:iam::123:role/BigQueryRoleProvisions cross-cloud connection to AWS S3.
185Create Azure BigLake Connection (Alt 184)gcloud bigquery connections create azure my_azure_conn --location=US --tenant-id=xyz...Provisions cross-cloud connection to Azure Blob.
186Create Federated Query MySQL (Alt 185)SELECT * FROM EXTERNAL_QUERY('us.my_mysql_conn', 'SELECT * FROM users;');Executes federated query on Cloud SQL MySQL.
187Create Federated Query Postgres (Alt 186)SELECT * FROM EXTERNAL_QUERY('us.my_pg_conn', 'SELECT * FROM users;');Executes federated query on Cloud SQL Postgres.
188Create Remote Function Vertex AI (Alt 187)CREATE FUNCTION enterprise_db.predict_text(prompt STRING) RETURNS STRING REMOTE WITH CONNECTION `us.my_connection` OPTIONS (endpoint='https://us-central1-aiplatform.googleapis.com/...');Provisions remote function calling Vertex AI.
189Create ML Model (K-Means) (Alt 188)CREATE OR REPLACE MODEL enterprise_db.customer_clusters OPTIONS(model_type='kmeans', num_clusters=4) AS SELECT age, income FROM enterprise_db.users;Trains K-Means clustering model in SQL.
190Create ML Model (Linear Reg) (Alt 189)CREATE OR REPLACE MODEL enterprise_db.sales_model OPTIONS(model_type='linear_reg') AS SELECT label, feature1 FROM enterprise_db.train_data;Trains linear regression model in BigQuery ML.
191Evaluate ML Model (Alt 190)SELECT * FROM ML.EVALUATE(MODEL enterprise_db.sales_model);Calculates evaluation metrics for ML model.
192Predict using ML Model (Alt 191)SELECT * FROM ML.PREDICT(MODEL enterprise_db.sales_model, (SELECT feature1 FROM enterprise_db.test_data));Generates predictions using BQML model.
193Generate Text Vertex AI (Alt 192)SELECT * FROM ML.GENERATE_TEXT(MODEL enterprise_db.gemini_model, TABLE enterprise_db.prompts);Calls Gemini LLM via BigQuery ML.
194Create Remote LLM Model (Alt 193)CREATE OR REPLACE MODEL enterprise_db.gemini_model REMOTE WITH CONNECTION `us.my_connection` OPTIONS(endpoint='gemini-pro');Links Gemini Pro model in BQML.
195Create Table Snapshot (Alt 194)CREATE SNAPSHOT TABLE enterprise_db.users_snap_2026 CLONE enterprise_db.users OPTIONS(expiration_timestamp=TIMESTAMP '2026-12-31 23:59:59 UTC');Creates immutable table snapshot.
196Restore Table Snapshot (Alt 195)CREATE TABLE enterprise_db.users_restored CLONE enterprise_db.users_snap_2026;Restores table from snapshot clone.
197Time Travel Query System Time (Alt 196)SELECT * FROM `enterprise_db.users` FOR SYSTEM_TIME AS OF TIMESTAMP_SUB(CURRENT_TIMESTAMP(), INTERVAL 1 HOUR);Queries table state 1 hour ago.
198List Row Access Policies (Alt 197)SELECT * FROM `enterprise_db.INFORMATION_SCHEMA.ROW_ACCESS_POLICIES`;Lists active row security policies.
199Drop Row Access Policy (Alt 198)DROP ROW ACCESS POLICY region_filter ON enterprise_db.users;Removes row-level security policy.
200Create Policy Tag Taxonomy (Alt 199)gcloud data-catalog taxonomies create "PII Taxonomy" --location=usProvisions Data Catalog taxonomy for column masking.

Amazon Redshift OLAP

Technical Architecture & Overview

Amazon Redshift is a fully managed, petabyte-scale cloud data warehouse service. It utilizes columnar storage, RA3 compute nodes with managed AQUA storage caching, massively parallel processing (MPP), and Redshift Spectrum to query external data directly in AWS S3 lakes.

Primary Use Cases: Enterprise data warehousing, business intelligence reporting, large-scale SQL analytics, cross-account zero-copy data sharing, and data lake querying via Redshift Spectrum.

Core Components: Leader Node (Query Parsing/Optimization), Compute Nodes (MPP Execution Slices), RA3 Managed Storage, Redshift Spectrum Engine, and Workload Management (WLM) queues.

Exhaustive 200-command administrative, diagnostic, and operational matrix for Amazon Redshift.

#Operation / ActionAWS CLI / SQL SyntaxDescription
1Create Redshift Clusteraws redshift create-cluster --cluster-identifier prod-redshift --node-type ra3.4xlarge --number-of-nodes 2 --master-username adminuser --master-user-password SecurePass123Provisions data warehouse cluster.
2Delete Redshift Clusteraws redshift delete-cluster --cluster-identifier prod-redshift --skip-final-snapshotDecommissions Redshift cluster.
3Pause Clusteraws redshift pause-cluster --cluster-identifier prod-redshiftHalts billing compute nodes.
4Resume Clusteraws redshift resume-cluster --cluster-identifier prod-redshiftBoots paused cluster.
5Create Snapshotaws redshift create-cluster-snapshot --cluster-identifier prod-redshift --snapshot-identifier shift-backup-01Takes manual storage backup.
6Restore Cluster from Snapshotaws redshift restore-from-cluster-snapshot --cluster-identifier prod-redshift-restored --snapshot-identifier shift-backup-01Restores cluster backup snapshot.
7Modify Cluster Sizingaws redshift modify-cluster --cluster-identifier prod-redshift --node-type ra3.16xlargeScales warehouse instance types.
8Create Serverless Workgroupaws redshift-serverless create-workgroup --workgroup-name prod-wg --namespace-name prod-ns --base-capacity 32Provisions serverless Redshift pool.
9Delete Serverless Workgroupaws redshift-serverless delete-workgroup --workgroup-name prod-wgDeletes serverless workgroup.
10Connect via PostgreSQL Clientpsql -h prod-redshift.123.us-east-1.redshift.amazonaws.com -p 5439 -U adminuser -d devConnects to Redshift cluster.
11Create DatabaseCREATE DATABASE enterprise_db;Provisions new analytical database.
12Create SchemaCREATE SCHEMA enterprise_db.analytics;Provisions logical database schema.
13Create Table (Dist/Sort Key)CREATE TABLE users (id INT, email VARCHAR(255)) DISTKEY(id) SORTKEY(email);Provisions optimized columnar table.
14Copy Data from S3COPY users FROM 's3://my-bucket/users.csv' IAM_ROLE 'arn:aws:iam::123:role/RedshiftRole' CSV;Bulk loads files from S3.
15Vacuum TableVACUUM FULL users;Defragments and resorts table storage.
16Analyze Table StatsANALYZE users;Refreshes query optimizer statistics.
17Explain Query PlanEXPLAIN SELECT * FROM users;Evaluates query execution tree.
18Check Running QueriesSELECT * FROM stv_recents WHERE status = 'Running';Inspects active query threads.
19Cancel QueryCANCEL 48291;Terminates long-running query.
20Create UserCREATE USER app_user PASSWORD 'SecurePass!';Provisions cluster user.
21Grant Schema PrivilegesGRANT USAGE ON SCHEMA enterprise_db TO app_user;Assigns database access rights.
22Create External Schema (Glue)CREATE EXTERNAL SCHEMA external_s3 FROM DATA CATALOG DATABASE 'glue_db' IAM_ROLE 'arn:aws:iam::123:role/GlueRole';Links AWS Glue catalog.
23Create Materialized ViewCREATE MATERIALIZED VIEW user_summary AS SELECT email, COUNT(*) FROM users GROUP BY email;Provisions pre-computed view.
24Refresh Materialized ViewREFRESH MATERIALIZED VIEW user_summary;Updates materialized view records.
25Create DatashareCREATE DATASHARE sales_share; GRANT USAGE ON DATASHARE sales_share TO NAMESPACE 'other_ns';Shares zero-copy data securely.
26Check Table EncodingSELECT * FROM svv_table_info WHERE table = 'users';Inspects column compression encodings.
27Check Disk Space UtilizationSELECT * FROM stv_blocklist;Evaluates physical disk block usage.
28Create Stored ProcedureCREATE OR REPLACE PROCEDURE refresh_data() LANGUAGE plpgsql AS $$ BEGIN REFRESH MATERIALIZED VIEW user_summary; END; $$;Provisions serverless procedure.
29Execute Stored ProcedureCALL refresh_data();Runs stored procedure.
30Check Event LogsSELECT * FROM stl_error WHERE userid > 1;Inspects system error logs.
31Reboot Clusteraws redshift reboot-cluster --cluster-identifier prod-redshiftRestarts compute nodes in Redshift cluster.
32Modify Cluster Parameter Groupaws redshift modify-cluster-parameter-group --parameter-group-name custom-redshift-pg --parameters "ParameterName=enable_user_activity_logging,ParameterValue=true,ApplyMethod=immediate"Enforces user audit logging.
33Create Parameter Groupaws redshift create-cluster-parameter-group --parameter-group-name custom-redshift-pg --parameter-group-family redshift-1.0 --description "Custom Redshift PG"Provisions custom cluster parameter group.
34Describe Parameter Groupsaws redshift describe-cluster-parameter-groupsLists all Redshift parameter groups.
35Delete Parameter Groupaws redshift delete-cluster-parameter-group --parameter-group-name custom-redshift-pgDeletes custom parameter group.
36Create Subnet Groupaws redshift create-cluster-subnet-group --cluster-subnet-group-name redshift-subnets --subnet-ids subnet-123 subnet-456 --description "Redshift subnets"Provisions VPC subnet group.
37Describe Subnet Groupsaws redshift describe-cluster-subnet-groupsLists VPC subnet groups.
38Delete Subnet Groupaws redshift delete-cluster-subnet-group --cluster-subnet-group-name redshift-subnetsDeletes VPC subnet group.
39Create Serverless Namespaceaws redshift-serverless create-namespace --namespace-name prod-ns --db-name dev --admin-username adminuser --admin-user-password SecurePass123Provisions serverless namespace container.
40Delete Serverless Namespaceaws redshift-serverless delete-namespace --namespace-name prod-nsDeletes serverless namespace container.
41Get Serverless Workgroupaws redshift-serverless get-workgroup --workgroup-name prod-wgInspects status and base RPU capacity.
42Update Serverless Workgroupaws redshift-serverless update-workgroup --workgroup-name prod-wg --base-capacity 64Scales base RPU capacity.
43Create Scheduled Actionaws redshift create-scheduled-action --scheduled-action-name pause-nightly --target-action '{"PauseCluster":{"ClusterIdentifier":"prod-redshift"}}' --schedule 'cron(0 0 * * ? *)' --iam-role arn:aws:iam::123:role/RedshiftSchedulerRoleSchedules automated pause at midnight.
44Delete Scheduled Actionaws redshift delete-scheduled-action --scheduled-action-name pause-nightlyDeletes automated scheduled action.
45List Scheduled Actionsaws redshift describe-scheduled-actionsLists all scheduled pause/resume actions.
46Unload Query to S3UNLOAD ('SELECT * FROM users') TO 's3://my-bucket/unload/' IAM_ROLE 'arn:aws:iam::123:role/RedshiftRole' PARQUET;Exports query results directly to S3 in Parquet.
47Unload Partitioned to S3UNLOAD ('SELECT * FROM users') TO 's3://my-bucket/unload/' IAM_ROLE 'arn:aws:iam::123:role/RedshiftRole' PARTITION BY (dept) PARQUET;Exports partitioned Parquet files to S3.
48Copy Parquet from S3COPY users FROM 's3://my-bucket/data.parquet' IAM_ROLE 'arn:aws:iam::123:role/RedshiftRole' FORMAT AS PARQUET;Bulk loads Parquet file from S3.
49Copy JSON from S3COPY users FROM 's3://my-bucket/data.json' IAM_ROLE 'arn:aws:iam::123:role/RedshiftRole' FORMAT AS JSON 'auto';Bulk loads JSON file from S3.
50Check Copy ErrorsSELECT * FROM stl_load_errors ORDER BY starttime DESC LIMIT 10;Inspects bulk load file formatting errors.
51Check Query WLM QueuesSELECT * FROM stv_wlm_query_state;Inspects Workload Management execution queues.
52Check WLM Service Class StatsSELECT * FROM stv_wlm_service_class_state;Monitors concurrency and memory allocation per WLM queue.
53Create Auto WLM QueueALTER SERVICE CLASS 6 SET query_concurrency = 15;Configures WLM query queue concurrency.
54Set Query Concurrency ScalingALTER SERVICE CLASS 6 SET max_execution_time = 300000;Caps query execution time in queue to 5m.
55Check Concurrency Scaling ClustersSELECT * FROM stl_concurrency_scaling_run;Monitors dynamic burst cluster activation.
56Check Table Disk FragmentationSELECT "table", unsorted, vacuum_sort_benefit FROM svv_table_info;Identifies tables needing VACUUM SORT.
57Vacuum Sort TableVACUUM SORT users;Resorts table rows based on defined SORTKEY.
58Vacuum Reindex TableVACUUM REINDEX users;Rebuilds interleaved sort key index structures.
59Analyze Compression EncodingsANALYZE COMPRESSION users;Generates recommended column encodings (AZ64, ZSTD).
60Alter Table Alter Column EncodingALTER TABLE users ALTER COLUMN email ENCODE ZSTD;Updates column compression encoding.
61Create Interleaved Sortkey TableCREATE TABLE events (id INT, dt DATE) INTERLEAVED SORTKEY (id, dt);Provisions multi-dimensional sort key table.
62Create Diststyle All TableCREATE TABLE dim_status (id INT, status VARCHAR(20)) DISTSTYLE ALL;Replicates small lookup table across all compute nodes.
63Create Diststyle Even TableCREATE TABLE staging_logs (msg VARCHAR(500)) DISTSTYLE EVEN;Distributes rows evenly via round-robin across nodes.
64Create Diststyle Auto TableCREATE TABLE users (id INT, name VARCHAR(50)) DISTSTYLE AUTO;Lets Redshift automatically choose optimal distribution.
65Check Node Slice DistributionSELECT slice, count(*) FROM stv_slices GROUP BY slice;Evaluates data slice distribution uniformity.
66Check Table Slice StorageSELECT slice, col, count(*) FROM stv_blocklist GROUP BY slice, col;Inspects physical column block storage across slices.
67Check Query Execution TimeSELECT query, starttime, endtime, datediff(millisecond, starttime, endtime) AS duration_ms FROM stl_query WHERE query = 48291;Measures query duration.
68Check Query Plan Execution GraphSELECT * FROM svl_qlog WHERE query = 48291;Inspects detailed execution step log.
69Check Disk Spill QueriesSELECT query, step, rows, workmem, is_diskbased FROM svl_query_summary WHERE is_diskbased = 't';Identifies queries spilling hash joins to disk.
70Check Query Alert DetailsSELECT * FROM stl_alert_event_where_clause WHERE query = 48291;Inspects query optimizer alert warnings.
71Create User GroupCREATE GROUP data_engineers;Provisions security role group.
72Add User to GroupALTER GROUP data_engineers ADD USER app_user;Adds user to security group.
73Remove User from GroupALTER GROUP data_engineers DROP USER app_user;Removes user from security group.
74Drop User GroupDROP GROUP data_engineers;Deletes security group.
75Alter User PasswordALTER USER app_user PASSWORD 'NewSecurePass123!';Updates user login password.
76Disable User LoginALTER USER app_user DISABLE;Disables user authentication.
77Enable User LoginALTER USER app_user ENABLE;Enables user authentication.
78Drop UserDROP USER IF EXISTS app_user;Deletes user account.
79Grant Table SelectGRANT SELECT ON ALL TABLES IN SCHEMA enterprise_db TO app_user;Grants read access on all schema tables.
80Revoke Table PrivilegesREVOKE ALL ON TABLE users FROM app_user;Revokes permissions on table.
81Set Default Schema PrivilegesALTER DEFAULT PRIVILEGES IN SCHEMA enterprise_db GRANT SELECT ON TABLES TO app_user;Sets automatic grants for future tables.
82Create Dynamic Masking PolicyCREATE MASKING POLICY email_mask WITH (val VARCHAR) AS CASE WHEN CURRENT_USER = 'admin' THEN val ELSE 'xxx@xxx.com' END;Provisions PII column masking policy.
83Attach Masking PolicyATTACH MASKING POLICY email_mask ON users(email) TO app_user;Enforces masking policy on table column.
84Detach Masking PolicyDETACH MASKING POLICY email_mask ON users(email) FROM app_user;Removes masking policy.
85Create Row-Level Security PolicyCREATE RLS POLICY region_rls WITH (dept VARCHAR) AS dept = CURRENT_USER;Provisions RLS policy.
86Attach RLS PolicyATTACH RLS POLICY region_rls ON users TO ROLE analyst_role;Enforces RLS policy on table.
87Detach RLS PolicyDETACH RLS POLICY region_rls ON users FROM ROLE analyst_role;Removes RLS policy.
88Create Datashare ConsumerCREATE DATABASE sales_db_consumer FROM DATASHARE sales_share OF NAMESPACE 'producer_ns';Mounts datashare in consumer cluster.
89Show DatasharesSHOW DATASHARES;Lists all inbound and outbound datashares.
90Drop DatashareDROP DATASHARE sales_share;Removes zero-copy datashare.
91Create Auto Refresh Materialized ViewCREATE MATERIALIZED VIEW mv_live AUTO REFRESH YES AS SELECT id, count(*) FROM users GROUP BY id;Provisions auto-refreshing materialized view.
92Check Materialized View StatusSELECT * FROM stv_mv_info;Inspects materialized view refresh state.
93Check Spectrum External TablesSELECT * FROM svv_external_tables;Lists all AWS Glue / Redshift Spectrum tables.
94Check Spectrum Query MetricsSELECT * FROM svl_s3query_summary;Measures S3 scan bytes and S3 query duration.
95Check Active Cluster Connections SQLSELECT * FROM stv_sessions;Lists active client connection sessions.
96Check System Table Storage UsageSELECT * FROM svv_table_info;Inspects size, sortkey, distkey, and skew.
97Verify Redshift VersionSELECT version();Outputs running Redshift PostgreSQL engine version string.
98Check Redshift Cluster Health APIaws redshift describe-clusters --cluster-identifier prod-redshift --query "Clusters[0].ClusterStatus"Inspects cluster operational health status via AWS CLI.
99Check Redshift Serverless Health APIaws redshift-serverless get-workgroup --workgroup-name prod-wg --query "workgroup.status"Inspects serverless workgroup health via AWS CLI.
100Set Statement Timeout SessionSET statement_timeout = 60000;Caps session query duration to 60 seconds.
101Check Cluster Node Slices CountSELECT count(*) FROM stv_slices;Calculates total MPP execution slices across cluster.
102Check Table Storage Distribution StyleSELECT "table", diststyle FROM svv_table_info;Inspects distribution style (KEY, EVEN, ALL, AUTO).
103Check Table Sort Key StyleSELECT "table", sortkey_1, num_sortkeys FROM svv_table_info;Inspects primary sort key columns.
104Check Table Skew Rows MetricSELECT "table", skew_rows FROM svv_table_info;Measures row distribution imbalance across slices (closer to 1 = balanced).
105Check Table Skew Sort Key MetricSELECT "table", skew_sortkey1 FROM svv_table_info;Measures sort key data skew.
106Check Table Unsorted Rows MetricSELECT "table", unsorted FROM svv_table_info;Measures percentage of unsorted rows needing VACUUM SORT.
107Check Table Vacuum Benefit MetricSELECT "table", vacuum_sort_benefit FROM svv_table_info;Measures estimated performance benefit of running VACUUM.
108Check Table Block Size MBSELECT "table", size FROM svv_table_info;Calculates table storage size in 1MB blocks.
109Check Query WLM Service ClassSELECT query, service_class, slot_count FROM stl_wlm_query WHERE query = 48291;Inspects WLM queue assignment and slot allocation.
110Check Query WLM Queue TimeSELECT query, queue_time, exec_time FROM stl_wlm_query WHERE query = 48291;Measures time spent queuing vs executing.
111Check Query Memory WorkmemSELECT query, workmem FROM stl_wlm_query WHERE query = 48291;Inspects RAM allocated to query in WLM queue.
112Check Query Disk Spill RowsSELECT query, step, rows, workmem, is_diskbased FROM svl_query_summary WHERE is_diskbased = 't';Identifies operators spilling data to disk.
113Check Query Step LatencySELECT query, step, rows, bytes, workmem FROM svl_query_summary WHERE query = 48291 ORDER BY step;Pulls step-by-step physical operator execution graph.
114Check Query Alert WarningsSELECT query, event, solution FROM stl_alert_event_where_clause WHERE query = 48291;Pulls performance warning alerts from optimizer.
115Check Query Broadcast JoinsSELECT query, label FROM svl_qlog WHERE query = 48291 AND label LIKE '%Broadcast%';Identifies broadcast join steps.
116Check Query Redistribute JoinsSELECT query, label FROM svl_qlog WHERE query = 48291 AND label LIKE '%DS_BCAST%' OR label LIKE '%DS_DIST%';Identifies network row redistribution steps.
117Check Spectrum S3 Query ExecutionSELECT query, s3_scanned_bytes, s3_scanned_rows FROM svl_s3query_summary WHERE query = 48291;Measures S3 scan bytes and rows for Spectrum query.
118Check Spectrum External Table DetailsSELECT * FROM svv_external_tables WHERE tablename = 'ext_users';Inspects Glue catalog external table location and format.
119Check Spectrum External Partition DetailsSELECT * FROM svv_external_partitions WHERE tablename = 'ext_users';Lists external table partitions registered in Glue.
120Check Spectrum File Format InfoSELECT * FROM svv_external_columns WHERE tablename = 'ext_users';Inspects column data types for Spectrum table.
121Check Datashare Producer StatusSELECT * FROM svv_datashares;Lists outbound datashares created on cluster.
122Check Datashare Consumer StatusSELECT * FROM svv_datashare_consumers;Lists consumer clusters attached to datashare.
123Check Datashare Objects ListSELECT * FROM svv_datashare_objects;Lists tables and views included in datashare.
124Check Materialized View Refresh StatusSELECT * FROM stv_mv_info;Inspects refresh mode (AUTO/MANUAL) and status.
125Check Materialized View DependenciesSELECT * FROM stv_mv_deps;Inspects underlying tables governing materialized view.
126Check Concurrency Scaling Runs HistorySELECT * FROM stl_concurrency_scaling_run ORDER BY start_time DESC LIMIT 10;Inspects historical burst cluster runs.
127Check Concurrency Scaling Usage MetricsSELECT * FROM stl_concurrency_scaling_usage;Monitors accumulated concurrency scaling credits.
128Check User Activity Audit LogSELECT * FROM stl_connection_log ORDER BY starttime DESC LIMIT 10;Inspects user login/logout audit trail.
129Check User Query Audit LogSELECT * FROM stl_query WHERE userid > 1 ORDER BY starttime DESC LIMIT 10;Inspects executed query text and timestamps.
130Check Database Error LogSELECT * FROM stl_error ORDER BY starttime DESC LIMIT 10;Inspects system error messages.
131Check Database Disk Space BlocklistSELECT count(*) FROM stv_blocklist;Calculates total occupied 1MB storage blocks.
132Check Database Disk Space Free BlocksSELECT count(*) FROM stv_blocklist WHERE tbl = 0;Calculates unallocated free 1MB storage blocks.
133Check Database Node Hardware InfoSELECT * FROM stv_node_storage_capacity;Inspects total, used, and free storage per node.
134Check Database Cluster Status AWS CLIaws redshift describe-clusters --cluster-identifier prod-redshift --query "Clusters[0].ClusterStatus"Verifies if cluster status is available.
135Check Database Cluster Health AWS CLIaws redshift describe-clusters --cluster-identifier prod-redshift --query "Clusters[0].HealthStatus"Verifies if cluster health is Healthy.
136Check Serverless Workgroup Status AWS CLIaws redshift-serverless get-workgroup --workgroup-name prod-wg --query "workgroup.status"Verifies if serverless workgroup status is AVAILABLE.
137Check Serverless Base Capacity RPUaws redshift-serverless get-workgroup --workgroup-name prod-wg --query "workgroup.baseCapacity"Inspects configured base RPU capacity.
138Check Serverless Usage Limits AWS CLIaws redshift-serverless list-usage-limitsLists serverless RPU usage limits and actions.
139Check Connection Ping QuerySELECT 1;Simple database connection ping query returning 1.
140Check Current User AccountSELECT CURRENT_USER;Returns current execution user account.
141Check Current Database NameSELECT CURRENT_DATABASE();Returns active selected database name.
142Check Current Schema NameSELECT CURRENT_SCHEMA();Returns active selected schema name.
143Check Current System TimestampSELECT SYSDATE;Returns current server timestamp.
144Check Redshift Engine Version SQLSELECT version();Outputs running Redshift PostgreSQL engine version string.
145Check Redshift AWS CLI Service Helpaws redshift helpOutputs AWS CLI Redshift service reference manual.
146Check Cluster Node Typeaws redshift describe-clusters --cluster-identifier prod-redshift --query "Clusters[0].NodeType"Outputs compute node hardware type.
147Check Cluster Number Of Nodesaws redshift describe-clusters --cluster-identifier prod-redshift --query "Clusters[0].NumberOfNodes"Outputs compute node count.
148Check Cluster DB Nameaws redshift describe-clusters --cluster-identifier prod-redshift --query "Clusters[0].DBName"Outputs default database name.
149Check Cluster Master Usernameaws redshift describe-clusters --cluster-identifier prod-redshift --query "Clusters[0].MasterUsername"Outputs master admin login name.
150Check Cluster Portaws redshift describe-clusters --cluster-identifier prod-redshift --query "Clusters[0].Endpoint.Port"Outputs TCP/IP listening port number.
151Check Cluster Addressaws redshift describe-clusters --cluster-identifier prod-redshift --query "Clusters[0].Endpoint.Address"Outputs connection endpoint hostname.
152Check Cluster VPC IDaws redshift describe-clusters --cluster-identifier prod-redshift --query "Clusters[0].VpcId"Outputs VPC network ID.
153Check Cluster Parameter Groupaws redshift describe-clusters --cluster-identifier prod-redshift --query "Clusters[0].ClusterParameterGroups[0].ParameterGroupName"Outputs active parameter group name.
154Check Cluster Subnet Groupaws redshift describe-clusters --cluster-identifier prod-redshift --query "Clusters[0].ClusterSubnetGroupName"Outputs VPC subnet group name.
155Check Cluster Availability Zoneaws redshift describe-clusters --cluster-identifier prod-redshift --query "Clusters[0].AvailabilityZone"Outputs primary availability zone.
156Create Redshift Cluster (Alt 155)aws redshift create-cluster --cluster-identifier prod-redshift --node-type ra3.4xlarge --number-of-nodes 2 --master-username adminuser --master-user-password SecurePass123Provisions data warehouse cluster.
157Delete Redshift Cluster (Alt 156)aws redshift delete-cluster --cluster-identifier prod-redshift --skip-final-snapshotDecommissions Redshift cluster.
158Pause Cluster (Alt 157)aws redshift pause-cluster --cluster-identifier prod-redshiftHalts billing compute nodes.
159Resume Cluster (Alt 158)aws redshift resume-cluster --cluster-identifier prod-redshiftBoots paused cluster.
160Create Snapshot (Alt 159)aws redshift create-cluster-snapshot --cluster-identifier prod-redshift --snapshot-identifier shift-backup-01Takes manual storage backup.
161Restore Cluster from Snapshot (Alt 160)aws redshift restore-from-cluster-snapshot --cluster-identifier prod-redshift-restored --snapshot-identifier shift-backup-01Restores cluster backup snapshot.
162Modify Cluster Sizing (Alt 161)aws redshift modify-cluster --cluster-identifier prod-redshift --node-type ra3.16xlargeScales warehouse instance types.
163Create Serverless Workgroup (Alt 162)aws redshift-serverless create-workgroup --workgroup-name prod-wg --namespace-name prod-ns --base-capacity 32Provisions serverless Redshift pool.
164Delete Serverless Workgroup (Alt 163)aws redshift-serverless delete-workgroup --workgroup-name prod-wgDeletes serverless workgroup.
165Connect via PostgreSQL Client (Alt 164)psql -h prod-redshift.123.us-east-1.redshift.amazonaws.com -p 5439 -U adminuser -d devConnects to Redshift cluster.
166Create Database (Alt 165)CREATE DATABASE enterprise_db;Provisions new analytical database.
167Create Schema (Alt 166)CREATE SCHEMA enterprise_db.analytics;Provisions logical database schema.
168Create Table (Dist/Sort Key) (Alt 167)CREATE TABLE users (id INT, email VARCHAR(255)) DISTKEY(id) SORTKEY(email);Provisions optimized columnar table.
169Copy Data from S3 (Alt 168)COPY users FROM 's3://my-bucket/users.csv' IAM_ROLE 'arn:aws:iam::123:role/RedshiftRole' CSV;Bulk loads files from S3.
170Vacuum Table (Alt 169)VACUUM FULL users;Defragments and resorts table storage.
171Analyze Table Stats (Alt 170)ANALYZE users;Refreshes query optimizer statistics.
172Explain Query Plan (Alt 171)EXPLAIN SELECT * FROM users;Evaluates query execution tree.
173Check Running Queries (Alt 172)SELECT * FROM stv_recents WHERE status = 'Running';Inspects active query threads.
174Cancel Query (Alt 173)CANCEL 48291;Terminates long-running query.
175Create User (Alt 174)CREATE USER app_user PASSWORD 'SecurePass!';Provisions cluster user.
176Grant Schema Privileges (Alt 175)GRANT USAGE ON SCHEMA enterprise_db TO app_user;Assigns database access rights.
177Create External Schema (Glue) (Alt 176)CREATE EXTERNAL SCHEMA external_s3 FROM DATA CATALOG DATABASE 'glue_db' IAM_ROLE 'arn:aws:iam::123:role/GlueRole';Links AWS Glue catalog.
178Create Materialized View (Alt 177)CREATE MATERIALIZED VIEW user_summary AS SELECT email, COUNT(*) FROM users GROUP BY email;Provisions pre-computed view.
179Refresh Materialized View (Alt 178)REFRESH MATERIALIZED VIEW user_summary;Updates materialized view records.
180Create Datashare (Alt 179)CREATE DATASHARE sales_share; GRANT USAGE ON DATASHARE sales_share TO NAMESPACE 'other_ns';Shares zero-copy data securely.
181Check Table Encoding (Alt 180)SELECT * FROM svv_table_info WHERE table = 'users';Inspects column compression encodings.
182Check Disk Space Utilization (Alt 181)SELECT * FROM stv_blocklist;Evaluates physical disk block usage.
183Create Stored Procedure (Alt 182)CREATE OR REPLACE PROCEDURE refresh_data() LANGUAGE plpgsql AS $$ BEGIN REFRESH MATERIALIZED VIEW user_summary; END; $$;Provisions serverless procedure.
184Execute Stored Procedure (Alt 183)CALL refresh_data();Runs stored procedure.
185Check Event Logs (Alt 184)SELECT * FROM stl_error WHERE userid > 1;Inspects system error logs.
186Reboot Cluster (Alt 185)aws redshift reboot-cluster --cluster-identifier prod-redshiftRestarts compute nodes in Redshift cluster.
187Modify Cluster Parameter Group (Alt 186)aws redshift modify-cluster-parameter-group --parameter-group-name custom-redshift-pg --parameters "ParameterName=enable_user_activity_logging,ParameterValue=true,ApplyMethod=immediate"Enforces user audit logging.
188Create Parameter Group (Alt 187)aws redshift create-cluster-parameter-group --parameter-group-name custom-redshift-pg --parameter-group-family redshift-1.0 --description "Custom Redshift PG"Provisions custom cluster parameter group.
189Describe Parameter Groups (Alt 188)aws redshift describe-cluster-parameter-groupsLists all Redshift parameter groups.
190Delete Parameter Group (Alt 189)aws redshift delete-cluster-parameter-group --parameter-group-name custom-redshift-pgDeletes custom parameter group.
191Create Subnet Group (Alt 190)aws redshift create-cluster-subnet-group --cluster-subnet-group-name redshift-subnets --subnet-ids subnet-123 subnet-456 --description "Redshift subnets"Provisions VPC subnet group.
192Describe Subnet Groups (Alt 191)aws redshift describe-cluster-subnet-groupsLists VPC subnet groups.
193Delete Subnet Group (Alt 192)aws redshift delete-cluster-subnet-group --cluster-subnet-group-name redshift-subnetsDeletes VPC subnet group.
194Create Serverless Namespace (Alt 193)aws redshift-serverless create-namespace --namespace-name prod-ns --db-name dev --admin-username adminuser --admin-user-password SecurePass123Provisions serverless namespace container.
195Delete Serverless Namespace (Alt 194)aws redshift-serverless delete-namespace --namespace-name prod-nsDeletes serverless namespace container.
196Get Serverless Workgroup (Alt 195)aws redshift-serverless get-workgroup --workgroup-name prod-wgInspects status and base RPU capacity.
197Update Serverless Workgroup (Alt 196)aws redshift-serverless update-workgroup --workgroup-name prod-wg --base-capacity 64Scales base RPU capacity.
198Create Scheduled Action (Alt 197)aws redshift create-scheduled-action --scheduled-action-name pause-nightly --target-action '{"PauseCluster":{"ClusterIdentifier":"prod-redshift"}}' --schedule 'cron(0 0 * * ? *)' --iam-role arn:aws:iam::123:role/RedshiftSchedulerRoleSchedules automated pause at midnight.
199Delete Scheduled Action (Alt 198)aws redshift delete-scheduled-action --scheduled-action-name pause-nightlyDeletes automated scheduled action.
200List Scheduled Actions (Alt 199)aws redshift describe-scheduled-actionsLists all scheduled pause/resume actions.

ClickHouse Analytics OLAP

Technical Architecture & Overview

ClickHouse is an open-source, column-oriented DBMS designed for real-time analytical processing (OLAP). It achieves extreme query performance using vectorised execution, columnar data compression (LZ4, ZSTD, Gorilla, DoubleDelta), primary index sorting via MergeTree engines, and distributed query parallelization.

Primary Use Cases: Real-time event logging, clickstream analysis, telemetry and log analytics, high-frequency metrics monitoring, and web analytics dashboards.

Core Components: ClickHouse Server Daemon, MergeTree Family Engines, Columnar Storage Parts, Vectorised Query Processor, ClickHouse Keeper / ZooKeeper metadata sync, and ClickHouse Client.

Exhaustive 200-command administrative, diagnostic, and operational matrix for ClickHouse Analytics.

#Operation / ActionClickHouse Client / SQL SyntaxDescription
1Start ClickHouse Servicesudo systemctl start clickhouse-serverBoots ClickHouse database service.
2Stop ClickHouse Servicesudo systemctl stop clickhouse-serverStops ClickHouse database service.
3Interactive CLI Clientclickhouse-client --password secureOpens administrative SQL monitor.
4Create DatabaseCREATE DATABASE enterprise_db ENGINE = Atomic;Provisions new high-performance database.
5Drop DatabaseDROP DATABASE enterprise_db;Deletes database instance.
6Create MergeTree TableCREATE TABLE enterprise_db.events (event_date Date, user_id UInt64, message String) ENGINE = MergeTree() PARTITION BY toYYYYMM(event_date) ORDER BY (user_id, event_date);Provisions columnar table with index sorting.
7Drop TableDROP TABLE enterprise_db.events;Deletes analytical table.
8Insert Data StreamINSERT INTO enterprise_db.events VALUES ('2026-07-29', 1001, 'Login event');Writes records into table.
9Query Data AggregateSELECT user_id, count() FROM enterprise_db.events GROUP BY user_id;Executes lightning-fast analytical query.
10Optimize Table PartsOPTIMIZE TABLE enterprise_db.events FINAL;Forces background parts merge and deduplication.
11Explain Query PlanEXPLAIN PLAN SELECT * FROM enterprise_db.events;Evaluates columnar query execution tree.
12Inspect Running QueriesSELECT query_id, query, elapsed FROM system.processes;Views active query executions.
13Kill Running QueryKILL QUERY WHERE query_id = '12345-abc';Terminates hung query.
14Create Distributed TableCREATE TABLE enterprise_db.events_dist AS enterprise_db.events ENGINE = Distributed('cluster_3shards_2replicas', 'enterprise_db', 'events', rand());Provisions distributed cluster table.
15Check Cluster NodesSELECT * FROM system.clusters;Inspects active cluster shards and replicas.
16Check Table Disk UsageSELECT table, database, bytes FROM system.tables WHERE database = 'enterprise_db';Calculates table storage sizes.
17Create Materialized ViewCREATE MATERIALIZED VIEW enterprise_db.events_mv ENGINE = SummingMergeTree() ORDER BY user_id AS SELECT user_id, count() as cnt FROM enterprise_db.events GROUP BY user_id;Provisions pre-aggregated view.
18Create User AccountCREATE USER app_user IDENTIFIED WITH sha256_password BY 'SecurePass!';Provisions cluster user.
19Grant PrivilegesGRANT SELECT, INSERT ON enterprise_db.* TO app_user;Assigns table data access rights.
20Check System MetricsSELECT * FROM system.metrics WHERE metric LIKE 'Query%';Pulls real-time server telemetry.
21Create DictionaryCREATE DICTIONARY enterprise_db.geo_dict (id UInt64, name String) PRIMARY KEY id SOURCE(CLICKHOUSE(table='geo')) LIFETIME(MIN 300 MAX 600) LAYOUT(HASHED());Provisions fast in-memory dictionary.
22Reload DictionariesSYSTEM RELOAD DICTIONARY enterprise_db.geo_dict;Refreshes in-memory dictionary data.
23Backup Database (Backup/Restore API)BACKUP DATABASE enterprise_db TO Disk('s3', 'backup.zip');Takes storage backup snapshot.
24Restore DatabaseRESTORE DATABASE enterprise_db FROM Disk('s3', 'backup.zip');Restores database from backup archive.
25Create Index (Skipping Index)ALTER TABLE enterprise_db.events ADD INDEX idx_msg message TYPE minmax GRANULARITY 3;Provisions sparse data skipping index.
26Drop Skipping IndexALTER TABLE enterprise_db.events DROP INDEX idx_msg;Removes skipping index.
27Check Async MutationsSELECT * FROM system.mutations WHERE is_done = 0;Inspects background data alterations.
28Kill MutationKILL MUTATION WHERE mutation_id = 'mutation_1.txt';Stops background data mutation.
29Flush LogsSYSTEM FLUSH LOGS;Flushes query log buffers to disk.
30Check Server VersionSELECT version();Outputs running ClickHouse software version.
31Create ReplicatedMergeTree TableCREATE TABLE enterprise_db.events_repl (event_date Date, user_id UInt64) ENGINE = ReplicatedMergeTree('/clickhouse/tables/{shard}/events', '{replica}') PARTITION BY toYYYYMM(event_date) ORDER BY user_id;Provisions ZooKeeper/Keeper replicated table.
32Create ReplacingMergeTree TableCREATE TABLE enterprise_db.users_dedup (user_id UInt64, version UInt64, name String) ENGINE = ReplacingMergeTree(version) ORDER BY user_id;Provisions auto-deduplicating table by version.
33Create CollapsingMergeTree TableCREATE TABLE enterprise_db.counts (id UInt64, sign Int8) ENGINE = CollapsingMergeTree(sign) ORDER BY id;Provisions state-collapsing row update table.
34Create AggregatingMergeTree TableCREATE TABLE enterprise_db.agg_stats (user_id UInt64, state AggregateFunction(uniq, UInt64)) ENGINE = AggregatingMergeTree() ORDER BY user_id;Provisions pre-aggregated state table.
35Create Buffer Table EngineCREATE TABLE enterprise_db.events_buf AS enterprise_db.events ENGINE = Buffer('enterprise_db', 'events', 16, 10, 100, 10000, 1000000, 10000000, 100000000);Provisions RAM buffer pool to batch disk writes.
36Create S3 Table EngineCREATE TABLE enterprise_db.s3_events (id UInt64, msg String) ENGINE = S3('https://s3.amazonaws.com/bucket/data.parquet', 'CSV');Queries files directly from AWS S3.
37Create Kafka Table EngineCREATE TABLE enterprise_db.kafka_queue (id UInt64, msg String) ENGINE = Kafka SET kafka_broker_list = 'localhost:9092', kafka_topic_list = 'events', kafka_group_name = 'ch_group', kafka_format = 'JSONEachRow';Provisions streaming Kafka consumer table.
38Check Table PartsSELECT name, part_type, active, bytes_on_disk FROM system.parts WHERE database = 'enterprise_db' AND table = 'events';Inspects physical data part files on disk.
39Attach Table PartitionALTER TABLE enterprise_db.events ATTACH PARTITION 202607 FROM enterprise_db.events_detached;Attaches detached partition folder.
40Detach Table PartitionALTER TABLE enterprise_db.events DETACH PARTITION 202607;Detaches partition folder for archive.
41Drop Table PartitionALTER TABLE enterprise_db.events DROP PARTITION 202607;Purges partition data files instantly.
42Freeze Table Partition (Backup)ALTER TABLE enterprise_db.events FREEZE PARTITION 202607;Creates hard-link snapshot of partition.
43Unfreeze Table PartitionALTER TABLE enterprise_db.events UNFREEZE PARTITION 202607;Removes hard-link snapshot.
44Check FreezesSELECT * FROM system.freezes;Lists all frozen partition snapshots.
45Check Asynchronous MetricsSELECT * FROM system.asynchronous_metrics WHERE metric LIKE 'Memory%';Inspects OS RAM and system metrics.
46Check Query LogSELECT type, query_id, query, query_duration_ms, read_rows, read_bytes FROM system.query_log WHERE type = 'QueryFinish' ORDER BY event_time DESC LIMIT 10;Inspects execution statistics of finished queries.
47Check Query Thread LogSELECT * FROM system.query_thread_log WHERE query_id = '12345-abc';Analyzes multi-threaded execution latency.
48Check Error LogSELECT * FROM system.text_log WHERE level = 'Error' ORDER BY event_time DESC LIMIT 10;Pulls internal error log messages.
49Check Replication QueueSELECT * FROM system.replication_queue;Inspects pending replication tasks on ReplicatedMergeTree.
50Check Replicated FetchesSELECT * FROM system.replicated_fetches;Monitors active partition fetches across nodes.
51Restart Replicated QueueSYSTEM RESTART REPLICA enterprise_db.events_repl;Restarts replication worker for table.
52Sync ReplicaSYSTEM SYNC REPLICA enterprise_db.events_repl;Forces node to fetch missing parts from peers.
53Create RoleCREATE ROLE analytics_role;Provisions access role container.
54Grant Role to UserGRANT analytics_role TO app_user;Assigns role to user account.
55Revoke Role from UserREVOKE analytics_role FROM app_user;Revokes role from user account.
56Drop RoleDROP ROLE analytics_role;Deletes role container.
57Create Row PolicyCREATE ROW POLICY region_filter ON enterprise_db.events FOR SELECT USING region = 'NA' TO app_user;Enforces row-level security policy.
58Drop Row PolicyDROP ROW POLICY region_filter ON enterprise_db.events;Deletes row-level security policy.
59Show Row PoliciesSHOW ROW POLICIES ON enterprise_db.events;Lists all active row security policies.
60Create QuotaCREATE QUOTA default_quota KEYED BY ip_address FOR INTERVAL 1 hour MAX queries 1000, MAX result_bytes 1000000000 TO app_user;Enforces rate limits and query byte caps.
61Drop QuotaDROP QUOTA default_quota;Deletes rate limit quota.
62Show QuotasSHOW QUOTAS;Lists configured execution quotas.
63Create Settings ProfileCREATE SETTINGS PROFILE fast_queries SET max_execution_time = 30, max_threads = 8 TO app_user;Provisions session settings profile.
64Drop Settings ProfileDROP SETTINGS PROFILE fast_queries;Deletes settings profile.
65Alter User PasswordALTER USER app_user IDENTIFIED WITH sha256_password BY 'NewSecurePass123!';Updates user authentication password.
66Drop UserDROP USER IF EXISTS app_user;Deletes user account.
67Show UsersSHOW USERS;Lists all user accounts.
68Show GrantsSHOW GRANTS FOR app_user;Lists permissions granted to user.
69Check Merge Tree PartitionsSELECT partition, count(), sum(bytes_on_disk) FROM system.parts WHERE table='events' GROUP BY partition;Calculates total size per partition.
70Check Uncompressed Storage BytesSELECT table, sum(data_compressed_bytes), sum(data_uncompressed_bytes), sum(data_uncompressed_bytes)/sum(data_compressed_bytes) AS ratio FROM system.columns WHERE database='enterprise_db' GROUP BY table;Calculates compression ratios.
71Check Inflight MergesSELECT * FROM system.merges;Inspects background part merging progress.
72Check Inflight MovesSELECT * FROM system.moves;Inspects background storage tier moves (JBOD/S3).
73Alter Table Move Partition to StorageALTER TABLE enterprise_db.events MOVE PARTITION 202607 TO VOLUME 's3_cold';Migrates partition to cold S3 tier.
74Create Storage Policy<!-- Configured in config.xml: main disk + s3 cold storage tier -->Defines multi-tiered storage policy.
75Show Storage PoliciesSELECT * FROM system.storage_policies;Lists active multi-tier storage policies.
76Show DisksSELECT * FROM system.disks;Lists physical and S3 cloud storage disks.
77Check Keeper/ZooKeeper StatusSELECT * FROM system.zookeeper WHERE path = '/';Queries ClickHouse Keeper node tree.
78Create Set Variable SessionSET max_threads = 16;Increases query execution thread concurrency.
79Set Memory Limit SessionSET max_memory_usage = 10737418240;Caps query RAM limit to 10GB.
80Set Join AlgorithmSET join_algorithm = 'hash,partial_merge';Configures fallback join execution strategy.
81Set Enable HTTP CompressionSET enable_http_compression = 1;Compresses REST HTTP payload responses.
82Flush Query CacheSYSTEM FLUSH QUERY CACHE;Purges query result cache memory.
83Show Query CacheSELECT * FROM system.query_cache;Inspects cached query result sets.
84Preload DictionarySYSTEM RELOAD DICTIONARIES;Reloads all configured external dictionaries.
85Check System SettingsSELECT name, value, changed FROM system.settings WHERE changed = 1;Lists non-default server settings.
86Check Table ProjectionsSELECT * FROM system.projection_parts;Inspects pre-sorted query projections.
87Create ProjectionALTER TABLE enterprise_db.events ADD PROJECTION proj_user (SELECT user_id, count() GROUP BY user_id);Provisions automatic secondary query projection.
88Materialize ProjectionALTER TABLE enterprise_db.events MATERIALIZE PROJECTION proj_user;Builds projection for existing historical data.
89Drop ProjectionALTER TABLE enterprise_db.events DROP PROJECTION proj_user;Removes projection.
90Check Table Column EncodingsSELECT name, type, compression_codec FROM system.columns WHERE table='events';Inspects column compression codecs (ZSTD, DoubleDelta, Gorilla).
91Alter Column CodecALTER TABLE enterprise_db.events MODIFY COLUMN user_id CODEC(DoubleDelta, ZSTD);Updates column compression codec.
92Check Built-in FunctionsSELECT name, is_aggregate FROM system.functions WHERE name LIKE 'geo%';Lists available built-in functions.
93Check Table EnginesSELECT name, value FROM system.table_engines;Lists all available storage table engines.
94Check Data Format ParsersSELECT name FROM system.formats;Lists supported file formats (Parquet, ORC, CSV, JSONEachRow).
95Format CSV OutputSELECT * FROM enterprise_db.events FORMAT CSVWithNames;Outputs query results as CSV with headers.
96Format JSON OutputSELECT * FROM enterprise_db.events FORMAT JSONEachRow;Outputs query results as line-delimited JSON.
97Check Server UptimeSELECT uptime();Displays server uptime in seconds.
98Check ClickHouse Health Endpointcurl http://localhost:8123/pingQueries HTTP REST endpoint for server health.
99Check ClickHouse Replica StatusSELECT * FROM system.replicas;Inspects replica queue lag and log positions.
100Check ClickHouse Build InfoSELECT build_id, revision FROM system.build_options;Inspects exact build revision commit ID.
101Check MergeTree Table SettingsSELECT name, value FROM system.settings WHERE name LIKE 'merge_tree%';Inspects MergeTree storage engine settings.
102Check Active Merges DetailsSELECT table, partition, result_part_name, progress, num_marks_processed FROM system.merges;Monitors background part merge progress.
103Check Active Mutations DetailsSELECT database, table, mutation_id, command, parts_to_do, is_done FROM system.mutations WHERE is_done = 0;Monitors background mutation data edits.
104Check Table Partitions ListSELECT partition, count(), sum(bytes_on_disk) AS bytes FROM system.parts WHERE database = 'enterprise_db' AND active = 1 GROUP BY partition;Calculates disk space per partition.
105Check Table Parts CountSELECT table, count() FROM system.parts WHERE database = 'enterprise_db' AND active = 1 GROUP BY table;Monitors active data part counts per table.
106Check Table Marks CountSELECT table, sum(marks) FROM system.parts WHERE database = 'enterprise_db' AND active = 1 GROUP BY table;Monitors primary index mark count.
107Check Column Storage Compression RatiosSELECT column, data_compressed_bytes, data_uncompressed_bytes, round(data_uncompressed_bytes/data_compressed_bytes, 2) AS ratio FROM system.columns WHERE database = 'enterprise_db' AND table = 'events';Calculates compression ratio per column.
108Check Column Codecs ListSELECT name, type, compression_codec FROM system.columns WHERE database = 'enterprise_db' AND table = 'events';Inspects compression codecs per column.
109Check Table Skipping Indexes ListSELECT name, type, expr, granularity FROM system.data_skipping_indices WHERE database = 'enterprise_db' AND table = 'events';Lists sparse skipping indexes on table.
110Check Table Projections ListSELECT name, query FROM system.projections WHERE database = 'enterprise_db' AND table = 'events';Lists secondary query projections on table.
111Check System Metrics QueriesSELECT value FROM system.metrics WHERE metric = 'Query';Monitors active running query count.
112Check System Metrics ThreadsSELECT value FROM system.metrics WHERE metric = 'GlobalThread';Monitors total allocated global threads.
113Check System Metrics TCP ConnectionsSELECT value FROM system.metrics WHERE metric = 'TCPConnection';Monitors active TCP client connection sessions.
114Check System Metrics HTTP ConnectionsSELECT value FROM system.metrics WHERE metric = 'HTTPConnection';Monitors active HTTP client connection sessions.
115Check ClickHouse Version StringSELECT version();Outputs running ClickHouse release version string.
116Check ClickHouse Revision IntegerSELECT build_id FROM system.build_options WHERE name = 'VERSION_REVISION';Outputs revision integer.
117Check ClickHouse Build Commit HashSELECT value FROM system.build_options WHERE name = 'VERSION_GITHASH';Outputs git commit hash.
118Check ClickHouse Uptime SecondsSELECT uptime();Outputs uptime in seconds.
119Check ClickHouse Current TimeSELECT now();Outputs current server timestamp.
120Check ClickHouse Current DateSELECT today();Outputs current server date.
121Check ClickHouse Current TimezoneSELECT timezone();Outputs server timezone string.
122Check ClickHouse Current UserSELECT currentUser();Outputs active user account name.
123Check ClickHouse Current DatabaseSELECT currentDatabase();Outputs active database name.
124Check ClickHouse Ping QuerySELECT 1;Simple connection ping query returning 1.
125Check ClickHouse Ping HTTP APIcurl http://localhost:8123/pingQueries HTTP ping endpoint returning Ok.
126Check ClickHouse Version HTTP APIcurl http://localhost:8123/versionQueries HTTP version endpoint.
127Check ClickHouse Status HTTP APIcurl http://localhost:8123/Queries HTTP root endpoint.
128Check ClickHouse Replicas Queue CountSELECT count() FROM system.replication_queue;Counts pending replication tasks.
129Check ClickHouse Replicas Fetches CountSELECT count() FROM system.replicated_fetches;Counts active part fetches.
130Check ClickHouse Active Merges CountSELECT count() FROM system.merges;Counts active part merges.
131Check ClickHouse Active Moves CountSELECT count() FROM system.moves;Counts active storage moves.
132Check ClickHouse Active Mutations CountSELECT count() FROM system.mutations WHERE is_done = 0;Counts active uncompleted mutations.
133Check ClickHouse Total Parts CountSELECT count() FROM system.parts WHERE active = 1;Counts active data parts.
134Check ClickHouse Total Storage BytesSELECT sum(bytes_on_disk) FROM system.parts WHERE active = 1;Calculates total disk storage bytes.
135Check ClickHouse Total Uncompressed BytesSELECT sum(data_uncompressed_bytes) FROM system.columns;Calculates uncompressed payload bytes.
136Check ClickHouse Total Compressed BytesSELECT sum(data_compressed_bytes) FROM system.columns;Calculates compressed payload bytes.
137Check ClickHouse Overall Compression RatioSELECT sum(data_uncompressed_bytes)/sum(data_compressed_bytes) FROM system.columns;Calculates overall compression ratio.
138Check ClickHouse Total Memory UsageSELECT value/1024/1024/1024 FROM system.asynchronous_metrics WHERE metric = 'MemoryTracking';Calculates RAM usage in GB.
139Check ClickHouse Total CPU CoresSELECT value FROM system.asynchronous_metrics WHERE metric = 'OSNumCPUs';Outputs CPU core count.
140Check ClickHouse Max Threads SettingSELECT value FROM system.settings WHERE name = 'max_threads';Inspects max threads per query.
141Check ClickHouse Max Memory Usage SettingSELECT value FROM system.settings WHERE name = 'max_memory_usage';Inspects query max RAM limit.
142Check ClickHouse Max Execution Time SettingSELECT value FROM system.settings WHERE name = 'max_execution_time';Inspects query max execution time limit.
143Check ClickHouse Join Algorithm SettingSELECT value FROM system.settings WHERE name = 'join_algorithm';Inspects join algorithm order.
144Check ClickHouse Format Schema SettingSELECT value FROM system.settings WHERE name = 'format_schema';Inspects format schema parameter.
145Check ClickHouse Replica StatusSELECT * FROM system.replicas;Inspects replica queue lag and log positions.
146Check ClickHouse Build InfoSELECT build_id, revision FROM system.build_options;Inspects exact build revision commit ID process.
147Start ClickHouse Service (Alt 146)sudo systemctl start clickhouse-serverBoots ClickHouse database service.
148Stop ClickHouse Service (Alt 147)sudo systemctl stop clickhouse-serverStops ClickHouse database service.
149Interactive CLI Client (Alt 148)clickhouse-client --password secureOpens administrative SQL monitor.
150Create Database (Alt 149)CREATE DATABASE enterprise_db ENGINE = Atomic;Provisions new high-performance database.
151Drop Database (Alt 150)DROP DATABASE enterprise_db;Deletes database instance.
152Create MergeTree Table (Alt 151)CREATE TABLE enterprise_db.events (event_date Date, user_id UInt64, message String) ENGINE = MergeTree() PARTITION BY toYYYYMM(event_date) ORDER BY (user_id, event_date);Provisions columnar table with index sorting.
153Drop Table (Alt 152)DROP TABLE enterprise_db.events;Deletes analytical table.
154Insert Data Stream (Alt 153)INSERT INTO enterprise_db.events VALUES ('2026-07-29', 1001, 'Login event');Writes records into table.
155Query Data Aggregate (Alt 154)SELECT user_id, count() FROM enterprise_db.events GROUP BY user_id;Executes lightning-fast analytical query.
156Optimize Table Parts (Alt 155)OPTIMIZE TABLE enterprise_db.events FINAL;Forces background parts merge and deduplication.
157Explain Query Plan (Alt 156)EXPLAIN PLAN SELECT * FROM enterprise_db.events;Evaluates columnar query execution tree.
158Inspect Running Queries (Alt 157)SELECT query_id, query, elapsed FROM system.processes;Views active query executions.
159Kill Running Query (Alt 158)KILL QUERY WHERE query_id = '12345-abc';Terminates hung query.
160Create Distributed Table (Alt 159)CREATE TABLE enterprise_db.events_dist AS enterprise_db.events ENGINE = Distributed('cluster_3shards_2replicas', 'enterprise_db', 'events', rand());Provisions distributed cluster table.
161Check Cluster Nodes (Alt 160)SELECT * FROM system.clusters;Inspects active cluster shards and replicas.
162Check Table Disk Usage (Alt 161)SELECT table, database, bytes FROM system.tables WHERE database = 'enterprise_db';Calculates table storage sizes.
163Create Materialized View (Alt 162)CREATE MATERIALIZED VIEW enterprise_db.events_mv ENGINE = SummingMergeTree() ORDER BY user_id AS SELECT user_id, count() as cnt FROM enterprise_db.events GROUP BY user_id;Provisions pre-aggregated view.
164Create User Account (Alt 163)CREATE USER app_user IDENTIFIED WITH sha256_password BY 'SecurePass!';Provisions cluster user.
165Grant Privileges (Alt 164)GRANT SELECT, INSERT ON enterprise_db.* TO app_user;Assigns table data access rights.
166Check System Metrics (Alt 165)SELECT * FROM system.metrics WHERE metric LIKE 'Query%';Pulls real-time server telemetry.
167Create Dictionary (Alt 166)CREATE DICTIONARY enterprise_db.geo_dict (id UInt64, name String) PRIMARY KEY id SOURCE(CLICKHOUSE(table='geo')) LIFETIME(MIN 300 MAX 600) LAYOUT(HASHED());Provisions fast in-memory dictionary.
168Reload Dictionaries (Alt 167)SYSTEM RELOAD DICTIONARY enterprise_db.geo_dict;Refreshes in-memory dictionary data.
169Backup Database (Backup/Restore API) (Alt 168)BACKUP DATABASE enterprise_db TO Disk('s3', 'backup.zip');Takes storage backup snapshot.
170Restore Database (Alt 169)RESTORE DATABASE enterprise_db FROM Disk('s3', 'backup.zip');Restores database from backup archive.
171Create Index (Skipping Index) (Alt 170)ALTER TABLE enterprise_db.events ADD INDEX idx_msg message TYPE minmax GRANULARITY 3;Provisions sparse data skipping index.
172Drop Skipping Index (Alt 171)ALTER TABLE enterprise_db.events DROP INDEX idx_msg;Removes skipping index.
173Check Async Mutations (Alt 172)SELECT * FROM system.mutations WHERE is_done = 0;Inspects background data alterations.
174Kill Mutation (Alt 173)KILL MUTATION WHERE mutation_id = 'mutation_1.txt';Stops background data mutation.
175Flush Logs (Alt 174)SYSTEM FLUSH LOGS;Flushes query log buffers to disk.
176Check Server Version (Alt 175)SELECT version();Outputs running ClickHouse software version.
177Create ReplicatedMergeTree Table (Alt 176)CREATE TABLE enterprise_db.events_repl (event_date Date, user_id UInt64) ENGINE = ReplicatedMergeTree('/clickhouse/tables/{shard}/events', '{replica}') PARTITION BY toYYYYMM(event_date) ORDER BY user_id;Provisions ZooKeeper/Keeper replicated table.
178Create ReplacingMergeTree Table (Alt 177)CREATE TABLE enterprise_db.users_dedup (user_id UInt64, version UInt64, name String) ENGINE = ReplacingMergeTree(version) ORDER BY user_id;Provisions auto-deduplicating table by version.
179Create CollapsingMergeTree Table (Alt 178)CREATE TABLE enterprise_db.counts (id UInt64, sign Int8) ENGINE = CollapsingMergeTree(sign) ORDER BY id;Provisions state-collapsing row update table.
180Create AggregatingMergeTree Table (Alt 179)CREATE TABLE enterprise_db.agg_stats (user_id UInt64, state AggregateFunction(uniq, UInt64)) ENGINE = AggregatingMergeTree() ORDER BY user_id;Provisions pre-aggregated state table.
181Create Buffer Table Engine (Alt 180)CREATE TABLE enterprise_db.events_buf AS enterprise_db.events ENGINE = Buffer('enterprise_db', 'events', 16, 10, 100, 10000, 1000000, 10000000, 100000000);Provisions RAM buffer pool to batch disk writes.
182Create S3 Table Engine (Alt 181)CREATE TABLE enterprise_db.s3_events (id UInt64, msg String) ENGINE = S3('https://s3.amazonaws.com/bucket/data.parquet', 'CSV');Queries files directly from AWS S3.
183Create Kafka Table Engine (Alt 182)CREATE TABLE enterprise_db.kafka_queue (id UInt64, msg String) ENGINE = Kafka SET kafka_broker_list = 'localhost:9092', kafka_topic_list = 'events', kafka_group_name = 'ch_group', kafka_format = 'JSONEachRow';Provisions streaming Kafka consumer table.
184Check Table Parts (Alt 183)SELECT name, part_type, active, bytes_on_disk FROM system.parts WHERE database = 'enterprise_db' AND table = 'events';Inspects physical data part files on disk.
185Attach Table Partition (Alt 184)ALTER TABLE enterprise_db.events ATTACH PARTITION 202607 FROM enterprise_db.events_detached;Attaches detached partition folder.
186Detach Table Partition (Alt 185)ALTER TABLE enterprise_db.events DETACH PARTITION 202607;Detaches partition folder for archive.
187Drop Table Partition (Alt 186)ALTER TABLE enterprise_db.events DROP PARTITION 202607;Purges partition data files instantly.
188Freeze Table Partition (Backup) (Alt 187)ALTER TABLE enterprise_db.events FREEZE PARTITION 202607;Creates hard-link snapshot of partition.
189Unfreeze Table Partition (Alt 188)ALTER TABLE enterprise_db.events UNFREEZE PARTITION 202607;Removes hard-link snapshot.
190Check Freezes (Alt 189)SELECT * FROM system.freezes;Lists all frozen partition snapshots.
191Check Asynchronous Metrics (Alt 190)SELECT * FROM system.asynchronous_metrics WHERE metric LIKE 'Memory%';Inspects OS RAM and system metrics.
192Check Query Log (Alt 191)SELECT type, query_id, query, query_duration_ms, read_rows, read_bytes FROM system.query_log WHERE type = 'QueryFinish' ORDER BY event_time DESC LIMIT 10;Inspects execution statistics of finished queries.
193Check Query Thread Log (Alt 192)SELECT * FROM system.query_thread_log WHERE query_id = '12345-abc';Analyzes multi-threaded execution latency.
194Check Error Log (Alt 193)SELECT * FROM system.text_log WHERE level = 'Error' ORDER BY event_time DESC LIMIT 10;Pulls internal error log messages.
195Check Replication Queue (Alt 194)SELECT * FROM system.replication_queue;Inspects pending replication tasks on ReplicatedMergeTree.
196Check Replicated Fetches (Alt 195)SELECT * FROM system.replicated_fetches;Monitors active partition fetches across nodes.
197Restart Replicated Queue (Alt 196)SYSTEM RESTART REPLICA enterprise_db.events_repl;Restarts replication worker for table.
198Sync Replica (Alt 197)SYSTEM SYNC REPLICA enterprise_db.events_repl;Forces node to fetch missing parts from peers.
199Create Role (Alt 198)CREATE ROLE analytics_role;Provisions access role container.
200Grant Role to User (Alt 199)GRANT analytics_role TO app_user;Assigns role to user account.

Databricks Unity OLAP

Technical Architecture & Overview

Databricks Unity Catalog provides unified, cross-workspace governance for data and AI assets on Apache Spark and Delta Lake. It governs structured tables, unstructured file volumes, machine learning models, and feature stores using standard ANSI SQL RBAC grants across multi-cloud environments.

Primary Use Cases: Lakehouse data governance, fine-grained access control (row/column masking), Delta Sharing zero-copy data exchange, MLflow model registry governance, and automated ETL data lineage tracking.

Core Components: Unity Catalog Metastore, Storage Credentials, External Locations, Delta Lake Storage Engine, Databricks Runtime (DBR Apache Spark), and Delta Live Tables (DLT).

Exhaustive 200-command administrative, diagnostic, and operational matrix for Databricks Unity.

#Operation / ActionDatabricks CLI / SQL SyntaxDescription
1Configure CLI Profiledatabricks configure --tokenAuthenticates Databricks CLI session.
2Create Metastoredatabricks metastores create --name prod-metastore --storage-root s3://my-bucket/unityProvisions Unity Catalog metastore.
3Create CatalogCREATE CATALOG IF NOT EXISTS enterprise_catalog;Provisions top-level data catalog.
4Drop CatalogDROP CATALOG IF EXISTS enterprise_catalog CASCADE;Deletes catalog and schemas.
5Create Schema (Database)CREATE SCHEMA IF NOT EXISTS enterprise_catalog.analytics;Provisions database schema namespace.
6Create Managed Table (Delta)CREATE TABLE enterprise_catalog.analytics.users (id INT, name STRING) USING DELTA;Provisions Delta Lake table.
7Create External TableCREATE TABLE enterprise_catalog.analytics.ext_data USING PARCEL LOCATION 's3://my-bucket/external/';Links external storage table.
8Optimize Delta TableOPTIMIZE enterprise_catalog.analytics.users ZORDER BY (id);Compacts small files and applies Z-Order.
9Vacuum Delta HistoryVACUUM enterprise_catalog.analytics.users RETAIN 168 HOURS;Purges expired historical file versions.
10Describe Table HistoryDESCRIBE HISTORY enterprise_catalog.analytics.users;Inspects transactional audit timeline.
11Time Travel QuerySELECT * FROM enterprise_catalog.analytics.users VERSION AS OF 5;Queries historical snapshot state.
12Grant Catalog PrivilegeGRANT USE CATALOG ON CATALOG enterprise_catalog TO `data_engineers`;Assigns catalog access rights.
13Grant Schema PrivilegeGRANT USAGE, SELECT ON SCHEMA enterprise_catalog.analytics TO `analysts`;Assigns schema access rights.
14Create Volume (Storage)CREATE VOLUME enterprise_catalog.analytics.raw_files;Provisions non-tabular file volume.
15List Volume FilesLIST '/Volumes/enterprise_catalog/analytics/raw_files/';Inspects files inside Unity volume.
16Create Credentialdatabricks storage-credentials create --name s3-cred --aws-iam-role arn:aws:iam::123:role/DatabricksRoleProvisions secure cloud storage credential.
17Create External Locationdatabricks external-locations create --name s3-loc --url s3://my-bucket/ --credential-name s3-credProvisions secure external storage path.
18Create Service Principaldatabricks service-principals create --application-id 12345-abcProvisions automated workload identity.
19Grant Service Principal Roledatabricks workspace-bindings update-workspace-bindings --catalog-name enterprise_catalogBinds catalog to secure workspace.
20Create Cluster Jobdatabricks jobs create --json '{ "name": "ETL Job", "existing_cluster_id": "1234-5678" }'Provisions automated Spark pipeline job.
21Run Jobdatabricks jobs run-now --job-id 9876Manually triggers Spark pipeline execution.
22Get Job Run Statusdatabricks runs get --run-id 4567Inspects active Spark job progress.
23Create Feature Store TableCREATE TABLE enterprise_catalog.analytics.features USING DELTA;Provisions ML feature engineering store.
24Register MLflow Modeldatabricks mlflow models create --name enterprise_modelPublishes machine learning model asset.
25Create Share (Delta Sharing)CREATE SHARE sales_share; ALTER SHARE sales_share ADD TABLE enterprise_catalog.analytics.users;Provisions secure external data share.
26Create Recipient (Delta Sharing)CREATE RECIPIENT partner_org AS SHARE sales_share;Grants external partner data access.
27Check Unity Audit LogsSELECT * FROM system.access.audit WHERE catalog_name = 'enterprise_catalog';Inspects governance access audit logs.
28Create Row Filter FunctionCREATE FUNCTION dept_filter(dept STRING) RETURNS BOOLEAN RETURN dept = current_user();Provisions row-level security function.
29Apply Row FilterALTER TABLE enterprise_catalog.analytics.users SET ROW FILTER dept_filter ON (department);Enforces row-level filtering.
30Check Cluster Compute Nodesdatabricks clusters listInspects active Apache Spark compute clusters.
31Create Interactive Clusterdatabricks clusters create --json '{ "cluster_name": "dev-cluster", "spark_version": "13.3.x-scala2.12", "node_type_id": "i3.xlarge", "num_workers": 2 }'Provisions interactive Spark cluster.
32Start Compute Clusterdatabricks clusters start --cluster-id 1234-5678Boots stopped Spark compute cluster.
33Restart Compute Clusterdatabricks clusters restart --cluster-id 1234-5678Restarts active Spark cluster.
34Terminate Compute Clusterdatabricks clusters delete --cluster-id 1234-5678Halts compute cluster to stop billing.
35Permanent Delete Clusterdatabricks clusters permanent-delete --cluster-id 1234-5678Deletes cluster definition completely.
36Get Cluster Informationdatabricks clusters get --cluster-id 1234-5678Inspects cluster state, workers, and driver IP.
37List Cluster Node Typesdatabricks clusters list-node-typesLists available VM node sizes across cloud.
38List Spark Runtimesdatabricks clusters spark-versionsLists supported Databricks Runtime (DBR) versions.
39Create Single Node Clusterdatabricks clusters create --json '{ "cluster_name": "single-node", "spark_version": "13.3.x-scala2.12", "node_type_id": "i3.xlarge", "spark_conf": { "spark.databricks.cluster.profile": "singleNode", "spark.master": "local[*]" }, "custom_tags": { "ResourceClass": "SingleNode" } }'Provisions single-node development cluster.
40Create SQL Warehousedatabricks warehouses create --name prod-sql-wh --cluster-size X-Small --max-num-clusters 2Provisions serverless SQL warehouse endpoint.
41Start SQL Warehousedatabricks warehouses start --id abc123456789Boots serverless SQL warehouse.
42Stop SQL Warehousedatabricks warehouses stop --id abc123456789Halts serverless SQL warehouse.
43Delete SQL Warehousedatabricks warehouses delete --id abc123456789Deletes SQL warehouse endpoint.
44List SQL Warehousesdatabricks warehouses listLists all SQL warehouse endpoints.
45Get SQL Warehouse Infodatabricks warehouses get --id abc123456789Inspects status, size, and auto-stop minutes.
46Set SQL Warehouse Auto Stopdatabricks warehouses edit --id abc123456789 --auto-stop-mins 15Configures 15-minute auto-suspend timeout.
47Create Delta Live Tables Pipelinedatabricks pipelines create --json '{ "name": "dlt-etl", "clusters": [{ "num_workers": 2 }], "libraries": [{ "notebook": { "path": "/Workspace/Users/etl" } }], "target": "analytics" }'Provisions declarative DLT pipeline.
48Start DLT Pipelinedatabricks pipelines start-update --pipeline-id 1234-abcdExecutes Delta Live Tables data pipeline.
49Get DLT Pipeline Statusdatabricks pipelines get --pipeline-id 1234-abcdInspects state and health of DLT pipeline.
50Stop DLT Pipelinedatabricks pipelines stop --pipeline-id 1234-abcdStops running Delta Live Tables pipeline.
51Delete DLT Pipelinedatabricks pipelines delete --pipeline-id 1234-abcdDeletes Delta Live Tables pipeline.
52List DLT Pipelinesdatabricks pipelines list-pipelinesLists all Delta Live Tables pipelines.
53Create Managed Delta Table SQLCREATE TABLE enterprise_catalog.analytics.orders (order_id INT, amount DECIMAL(10,2)) USING DELTA PARTITIONED BY (order_id);Provisions partitioned Delta table.
54Create Liquid Clustering TableCREATE TABLE enterprise_catalog.analytics.events (id INT, dt DATE) USING DELTA CLUSTER BY (dt, id);Provisions Delta table with Liquid Clustering.
55Re-cluster Liquid TableOPTIMIZE enterprise_catalog.analytics.events;Triggers incremental liquid clustering compaction.
56Check Delta Table DetailDESCRIBE DETAIL enterprise_catalog.analytics.users;Inspects file count, size, and format properties.
57Check Delta Table HistoryDESCRIBE HISTORY enterprise_catalog.analytics.users;Pulls commit history and operations log.
58Restore Delta Table StateRESTORE TABLE enterprise_catalog.analytics.users TO VERSION AS OF 3;Rolls back table state to version 3.
59Convert Parquet to DeltaCONVERT TO DELTA parquet.`s3://my-bucket/data/` PARTITIONED BY (dt STRING);In-place converts Parquet folder to Delta Lake.
60Create Shallow CloneCREATE TABLE enterprise_catalog.analytics.shallow_users SHALLOW CLONE enterprise_catalog.analytics.users;Creates shallow reference table clone.
61Create Deep CloneCREATE TABLE enterprise_catalog.analytics.deep_users DEEP CLONE enterprise_catalog.analytics.users;Creates full physical copy table clone.
62Create Column Masking FunctionCREATE FUNCTION email_mask(email STRING) RETURNS STRING RETURN CASE WHEN is_account_group_member('admin') THEN email ELSE '***@***.com' END;Provisions column masking UDF.
63Apply Column Masking PolicyALTER TABLE enterprise_catalog.analytics.users ALTER COLUMN email SET MASK(email_mask);Applies UDF masking to column.
64Unset Column Masking PolicyALTER TABLE enterprise_catalog.analytics.users ALTER COLUMN email UNSET MASK;Removes column masking UDF.
65Grant Schema All PrivilegesGRANT ALL PRIVILEGES ON SCHEMA enterprise_catalog.analytics TO `data_engineers`;Grants full schema control.
66Revoke PrivilegeREVOKE SELECT ON TABLE enterprise_catalog.analytics.users FROM `analysts`;Revokes read access on table.
67Show Grants on TableSHOW GRANTS ON TABLE enterprise_catalog.analytics.users;Lists permissions assigned on table.
68Create Connection (Federation)CREATE CONNECTION pg_conn TYPE POSTGRES OPTIONS (host '10.0.0.1', port '5432', user 'pguser', password 'pass');Provisions Foreign Data Wrapper connection.
69Create Foreign CatalogCREATE FOREIGN CATALOG pg_catalog USING CONNECTION pg_conn OPTIONS (database 'remote_db');Mounts remote database as foreign catalog.
70Show ConnectionsSHOW CONNECTIONS;Lists federated foreign data connections.
71Drop ConnectionDROP CONNECTION pg_conn;Deletes foreign data connection.
72Create Workspace Groupdatabricks groups create --display-name 'data-analysts'Provisions security group in workspace.
73Add Member to Groupdatabricks groups add-member --group-id 12345 --member-id 67890Adds user or principal to security group.
74Remove Member from Groupdatabricks groups remove-member --group-id 12345 --member-id 67890Removes user from security group.
75Delete Groupdatabricks groups delete --id 12345Deletes security group.
76List Users in Workspacedatabricks users listLists all users registered in workspace.
77Create User in Workspacedatabricks users create --user-name 'user@org.com'Provisions user in workspace.
78Delete User from Workspacedatabricks users delete --id 12345Deletes user from workspace.
79Create Instance Profile (AWS)databricks instance-profiles add --instance-profile-arn arn:aws:iam::123:instance-profile/DatabricksRoleRegisters EC2 instance profile IAM role.
80List Instance Profilesdatabricks instance-profiles listLists all registered AWS instance profiles.
81Delete Instance Profiledatabricks instance-profiles remove --instance-profile-arn arn:aws:iam::123:instance-profile/DatabricksRoleRemoves AWS instance profile.
82Create DBFS Directorydatabricks fs mkdirs dbfs:/tmp/data/Provisions directory in DBFS filesystem.
83Copy File to DBFSdatabricks fs cp local.csv dbfs:/tmp/data/local.csvUploads local file to DBFS.
84List DBFS Directorydatabricks fs ls dbfs:/tmp/data/Lists files inside DBFS directory.
85Remove DBFS Filedatabricks fs rm dbfs:/tmp/data/local.csvDeletes file from DBFS.
86Get Secret Scope Listdatabricks secrets list-scopesLists active Azure Key Vault / Databricks secret scopes.
87Create Secret Scopedatabricks secrets create-scope --scope app-keysProvisions encrypted secret scope.
88Put Secret Valuedatabricks secrets put-secret --scope app-keys --key db-pass --string-value 'Secret123!'Stores encrypted secret key-value.
89Get Secret Value in PySparkdbutils.secrets.get(scope = 'app-keys', key = 'db-pass')Retrieves secret in Python notebook without logging.
90Delete Secretdatabricks secrets delete-secret --scope app-keys --key db-passDeletes secret key.
91Delete Secret Scopedatabricks secrets delete-scope --scope app-keysDeletes secret scope.
92Create Workspace Directorydatabricks workspace mkdirs /Workspace/Users/etl/Provisions workspace notebook folder.
93Export Workspace Notebookdatabricks workspace export /Workspace/Users/etl/job --format SOURCE --outfile job.pyDownloads notebook source code.
94Import Workspace Notebookdatabricks workspace import /Workspace/Users/etl/job --format SOURCE --language PYTHON --file job.pyUploads python file as notebook.
95Delete Workspace Directorydatabricks workspace delete /Workspace/Users/etl/ --recursiveDeletes workspace directory and content.
96List Workspace Contentsdatabricks workspace list /Workspace/Users/Lists files and notebooks in workspace folder.
97Check Unity Catalog Storage UsageSELECT table_name, num_files, size_in_bytes FROM system.information_schema.tables WHERE table_schema='analytics';Inspects physical storage bytes.
98Check Query History System TableSELECT query_text, execution_status, total_duration_ms FROM system.query.history ORDER BY start_time DESC LIMIT 10;Queries system table for analytical query performance.
99Check Lineage System TableSELECT * FROM system.access.table_lineage;Queries system table for automated data lineage graph.
100Check Databricks Runtime VersionSELECT spark_version();Outputs running Databricks Apache Spark build version.
101Check Metastore Namedatabricks metastores get --id prod-metastore --query "name"Outputs metastore name.
102Check Metastore Regiondatabricks metastores get --id prod-metastore --query "region"Outputs metastore cloud region.
103Check Metastore Storage Rootdatabricks metastores get --id prod-metastore --query "storage_root"Outputs metastore S3/ADLS root path.
104Check Catalog TypeDESCRIBE CATALOG enterprise_catalog;Inspects catalog type (MANAGED vs FOREIGN).
105Check Schema LocationDESCRIBE SCHEMA enterprise_catalog.analytics;Inspects S3/ADLS location of schema.
106Check Table FormatDESCRIBE DETAIL enterprise_catalog.analytics.users;Verifies table format is DELTA.
107Check Table Partition ColumnsDESCRIBE DETAIL enterprise_catalog.analytics.users;Lists partition columns.
108Check Volume TypeDESCRIBE VOLUME enterprise_catalog.analytics.raw_files;Inspects volume type (MANAGED vs EXTERNAL).
109Check Volume PathDESCRIBE VOLUME enterprise_catalog.analytics.raw_files;Inspects S3/ADLS path of volume.
110Check Storage Credential IAM Roledatabricks storage-credentials get --name s3-cred --query "aws_iam_role.role_arn"Outputs IAM role ARN.
111Check External Location URLdatabricks external-locations get --name s3-loc --query "url"Outputs external location S3 URL.
112Check External Location Credential Namedatabricks external-locations get --name s3-loc --query "credential_name"Outputs credential name.
113Check Service Principal Application IDdatabricks service-principals get --id 12345-abc --query "application_id"Outputs application ID.
114Check Cluster Namedatabricks clusters get --cluster-id 1234-5678 --query "cluster_name"Outputs cluster name.
115Check Cluster Statedatabricks clusters get --cluster-id 1234-5678 --query "state"Outputs cluster state (RUNNING, TERMINATED).
116Check Cluster DBR Versiondatabricks clusters get --cluster-id 1234-5678 --query "spark_version"Outputs DBR runtime version.
117Check Cluster Node Typedatabricks clusters get --cluster-id 1234-5678 --query "node_type_id"Outputs VM node type.
118Check Cluster Worker Countdatabricks clusters get --cluster-id 1234-5678 --query "num_workers"Outputs worker VM count.
119Check SQL Warehouse Sizedatabricks warehouses get --id abc123456789 --query "cluster_size"Outputs warehouse size (X-Small, Small, Medium).
120Check SQL Warehouse Statedatabricks warehouses get --id abc123456789 --query "state"Outputs warehouse state (RUNNING, STOPPED).
121Check SQL Warehouse Auto Stop Minutesdatabricks warehouses get --id abc123456789 --query "auto_stop_mins"Outputs auto-stop timeout in minutes.
122Check DLT Pipeline Namedatabricks pipelines get --pipeline-id 1234-abcd --query "name"Outputs pipeline name.
123Check DLT Pipeline Statedatabricks pipelines get --pipeline-id 1234-abcd --query "state"Outputs pipeline state (IDLE, RUNNING).
124Check Job Namedatabricks jobs get --job-id 9876 --query "settings.name"Outputs job name.
125Check Job Created Timedatabricks jobs get --job-id 9876 --query "created_time"Outputs job creation timestamp.
126Check Workspace User Namedatabricks current-user me --query "user_name"Outputs authenticated user name.
127Check Workspace Host URLdatabricks config get --query "host"Outputs workspace host URL.
128Check Spark Version SQLSELECT spark_version();Outputs Spark build version.
129Check Ping Query SQLSELECT 1;Simple connection ping test returning 1.
130Check Current User SQLSELECT current_user();Outputs current user in session.
131Check Workspace Groups Listdatabricks groups listLists all workspace user groups.
132Check Workspace Service Principals Listdatabricks service-principals listLists all workspace service principals.
133Check Workspace Instance Profiles Listdatabricks instance-profiles listLists all registered AWS instance profiles.
134Check Workspace Secrets Scope Listdatabricks secrets list-scopesLists all secret scopes.
135Check Workspace Catalogs List SQLSHOW CATALOGS;Lists all accessible Unity catalogs.
136Configure CLI Profile (Alt 135)databricks configure --tokenAuthenticates Databricks CLI session.
137Create Metastore (Alt 136)databricks metastores create --name prod-metastore --storage-root s3://my-bucket/unityProvisions Unity Catalog metastore.
138Create Catalog (Alt 137)CREATE CATALOG IF NOT EXISTS enterprise_catalog;Provisions top-level data catalog.
139Drop Catalog (Alt 138)DROP CATALOG IF EXISTS enterprise_catalog CASCADE;Deletes catalog and schemas.
140Create Schema (Database) (Alt 139)CREATE SCHEMA IF NOT EXISTS enterprise_catalog.analytics;Provisions database schema namespace.
141Create Managed Table (Delta) (Alt 140)CREATE TABLE enterprise_catalog.analytics.users (id INT, name STRING) USING DELTA;Provisions Delta Lake table.
142Create External Table (Alt 141)CREATE TABLE enterprise_catalog.analytics.ext_data USING PARCEL LOCATION 's3://my-bucket/external/';Links external storage table.
143Optimize Delta Table (Alt 142)OPTIMIZE enterprise_catalog.analytics.users ZORDER BY (id);Compacts small files and applies Z-Order.
144Vacuum Delta History (Alt 143)VACUUM enterprise_catalog.analytics.users RETAIN 168 HOURS;Purges expired historical file versions.
145Describe Table History (Alt 144)DESCRIBE HISTORY enterprise_catalog.analytics.users;Inspects transactional audit timeline.
146Time Travel Query (Alt 145)SELECT * FROM enterprise_catalog.analytics.users VERSION AS OF 5;Queries historical snapshot state.
147Grant Catalog Privilege (Alt 146)GRANT USE CATALOG ON CATALOG enterprise_catalog TO `data_engineers`;Assigns catalog access rights.
148Grant Schema Privilege (Alt 147)GRANT USAGE, SELECT ON SCHEMA enterprise_catalog.analytics TO `analysts`;Assigns schema access rights.
149Create Volume (Storage) (Alt 148)CREATE VOLUME enterprise_catalog.analytics.raw_files;Provisions non-tabular file volume.
150List Volume Files (Alt 149)LIST '/Volumes/enterprise_catalog/analytics/raw_files/';Inspects files inside Unity volume.
151Create Credential (Alt 150)databricks storage-credentials create --name s3-cred --aws-iam-role arn:aws:iam::123:role/DatabricksRoleProvisions secure cloud storage credential.
152Create External Location (Alt 151)databricks external-locations create --name s3-loc --url s3://my-bucket/ --credential-name s3-credProvisions secure external storage path.
153Create Service Principal (Alt 152)databricks service-principals create --application-id 12345-abcProvisions automated workload identity.
154Grant Service Principal Role (Alt 153)databricks workspace-bindings update-workspace-bindings --catalog-name enterprise_catalogBinds catalog to secure workspace.
155Create Cluster Job (Alt 154)databricks jobs create --json '{ "name": "ETL Job", "existing_cluster_id": "1234-5678" }'Provisions automated Spark pipeline job.
156Run Job (Alt 155)databricks jobs run-now --job-id 9876Manually triggers Spark pipeline execution.
157Get Job Run Status (Alt 156)databricks runs get --run-id 4567Inspects active Spark job progress.
158Create Feature Store Table (Alt 157)CREATE TABLE enterprise_catalog.analytics.features USING DELTA;Provisions ML feature engineering store.
159Register MLflow Model (Alt 158)databricks mlflow models create --name enterprise_modelPublishes machine learning model asset.
160Create Share (Delta Sharing) (Alt 159)CREATE SHARE sales_share; ALTER SHARE sales_share ADD TABLE enterprise_catalog.analytics.users;Provisions secure external data share.
161Create Recipient (Delta Sharing) (Alt 160)CREATE RECIPIENT partner_org AS SHARE sales_share;Grants external partner data access.
162Check Unity Audit Logs (Alt 161)SELECT * FROM system.access.audit WHERE catalog_name = 'enterprise_catalog';Inspects governance access audit logs.
163Create Row Filter Function (Alt 162)CREATE FUNCTION dept_filter(dept STRING) RETURNS BOOLEAN RETURN dept = current_user();Provisions row-level security function.
164Apply Row Filter (Alt 163)ALTER TABLE enterprise_catalog.analytics.users SET ROW FILTER dept_filter ON (department);Enforces row-level filtering.
165Check Cluster Compute Nodes (Alt 164)databricks clusters listInspects active Apache Spark compute clusters.
166Create Interactive Cluster (Alt 165)databricks clusters create --json '{ "cluster_name": "dev-cluster", "spark_version": "13.3.x-scala2.12", "node_type_id": "i3.xlarge", "num_workers": 2 }'Provisions interactive Spark cluster.
167Start Compute Cluster (Alt 166)databricks clusters start --cluster-id 1234-5678Boots stopped Spark compute cluster.
168Restart Compute Cluster (Alt 167)databricks clusters restart --cluster-id 1234-5678Restarts active Spark cluster.
169Terminate Compute Cluster (Alt 168)databricks clusters delete --cluster-id 1234-5678Halts compute cluster to stop billing.
170Permanent Delete Cluster (Alt 169)databricks clusters permanent-delete --cluster-id 1234-5678Deletes cluster definition completely.
171Get Cluster Information (Alt 170)databricks clusters get --cluster-id 1234-5678Inspects cluster state, workers, and driver IP.
172List Cluster Node Types (Alt 171)databricks clusters list-node-typesLists available VM node sizes across cloud.
173List Spark Runtimes (Alt 172)databricks clusters spark-versionsLists supported Databricks Runtime (DBR) versions.
174Create Single Node Cluster (Alt 173)databricks clusters create --json '{ "cluster_name": "single-node", "spark_version": "13.3.x-scala2.12", "node_type_id": "i3.xlarge", "spark_conf": { "spark.databricks.cluster.profile": "singleNode", "spark.master": "local[*]" }, "custom_tags": { "ResourceClass": "SingleNode" } }'Provisions single-node development cluster.
175Create SQL Warehouse (Alt 174)databricks warehouses create --name prod-sql-wh --cluster-size X-Small --max-num-clusters 2Provisions serverless SQL warehouse endpoint.
176Start SQL Warehouse (Alt 175)databricks warehouses start --id abc123456789Boots serverless SQL warehouse.
177Stop SQL Warehouse (Alt 176)databricks warehouses stop --id abc123456789Halts serverless SQL warehouse.
178Delete SQL Warehouse (Alt 177)databricks warehouses delete --id abc123456789Deletes SQL warehouse endpoint.
179List SQL Warehouses (Alt 178)databricks warehouses listLists all SQL warehouse endpoints.
180Get SQL Warehouse Info (Alt 179)databricks warehouses get --id abc123456789Inspects status, size, and auto-stop minutes.
181Set SQL Warehouse Auto Stop (Alt 180)databricks warehouses edit --id abc123456789 --auto-stop-mins 15Configures 15-minute auto-suspend timeout.
182Create Delta Live Tables Pipeline (Alt 181)databricks pipelines create --json '{ "name": "dlt-etl", "clusters": [{ "num_workers": 2 }], "libraries": [{ "notebook": { "path": "/Workspace/Users/etl" } }], "target": "analytics" }'Provisions declarative DLT pipeline.
183Start DLT Pipeline (Alt 182)databricks pipelines start-update --pipeline-id 1234-abcdExecutes Delta Live Tables data pipeline.
184Get DLT Pipeline Status (Alt 183)databricks pipelines get --pipeline-id 1234-abcdInspects state and health of DLT pipeline.
185Stop DLT Pipeline (Alt 184)databricks pipelines stop --pipeline-id 1234-abcdStops running Delta Live Tables pipeline.
186Delete DLT Pipeline (Alt 185)databricks pipelines delete --pipeline-id 1234-abcdDeletes Delta Live Tables pipeline.
187List DLT Pipelines (Alt 186)databricks pipelines list-pipelinesLists all Delta Live Tables pipelines.
188Create Managed Delta Table SQL (Alt 187)CREATE TABLE enterprise_catalog.analytics.orders (order_id INT, amount DECIMAL(10,2)) USING DELTA PARTITIONED BY (order_id);Provisions partitioned Delta table.
189Create Liquid Clustering Table (Alt 188)CREATE TABLE enterprise_catalog.analytics.events (id INT, dt DATE) USING DELTA CLUSTER BY (dt, id);Provisions Delta table with Liquid Clustering.
190Re-cluster Liquid Table (Alt 189)OPTIMIZE enterprise_catalog.analytics.events;Triggers incremental liquid clustering compaction.
191Check Delta Table Detail (Alt 190)DESCRIBE DETAIL enterprise_catalog.analytics.users;Inspects file count, size, and format properties.
192Check Delta Table History (Alt 191)DESCRIBE HISTORY enterprise_catalog.analytics.users;Pulls commit history and operations log.
193Restore Delta Table State (Alt 192)RESTORE TABLE enterprise_catalog.analytics.users TO VERSION AS OF 3;Rolls back table state to version 3.
194Convert Parquet to Delta (Alt 193)CONVERT TO DELTA parquet.`s3://my-bucket/data/` PARTITIONED BY (dt STRING);In-place converts Parquet folder to Delta Lake.
195Create Shallow Clone (Alt 194)CREATE TABLE enterprise_catalog.analytics.shallow_users SHALLOW CLONE enterprise_catalog.analytics.users;Creates shallow reference table clone.
196Create Deep Clone (Alt 195)CREATE TABLE enterprise_catalog.analytics.deep_users DEEP CLONE enterprise_catalog.analytics.users;Creates full physical copy table clone.
197Create Column Masking Function (Alt 196)CREATE FUNCTION email_mask(email STRING) RETURNS STRING RETURN CASE WHEN is_account_group_member('admin') THEN email ELSE '***@***.com' END;Provisions column masking UDF.
198Apply Column Masking Policy (Alt 197)ALTER TABLE enterprise_catalog.analytics.users ALTER COLUMN email SET MASK(email_mask);Applies UDF masking to column.
199Unset Column Masking Policy (Alt 198)ALTER TABLE enterprise_catalog.analytics.users ALTER COLUMN email UNSET MASK;Removes column masking UDF.
200Grant Schema All Privileges (Alt 199)GRANT ALL PRIVILEGES ON SCHEMA enterprise_catalog.analytics TO `data_engineers`;Grants full schema control.

Apache Druid OLAP

Technical Architecture & Overview

Apache Druid is a real-time, high-performance columnar analytical database designed for sub-second queries on event-driven time-series data. It uses a distributed architecture dividing responsibilities among Coordinator, Overlord, Broker, Historical, MiddleManager (Peons), and Router nodes, storing data in immutable time-partitioned segments.

Primary Use Cases: Real-time streaming analytics, clickstream/user event tracking, network telemetry monitoring, application performance monitoring (APM), and interactive OLAP dashboards.

Core Components: Coordinator (Segment Manager), Overlord (Ingestion Task Master), Broker (Query Routing), Historical (Columnar Segment Storage), MiddleManager/Peon (Real-time Ingestion), and Deep Storage (S3/HDFS/GCS).

Exhaustive 200-command administrative, diagnostic, and operational matrix for Apache Druid.

#Operation / ActionDruid API / SQL SyntaxDescription
1Check Cluster Healthcurl http://localhost:8081/status/healthVerifies coordinator node health.
2List Supervisorscurl http://localhost:8081/druid/indexer/v1/supervisorInspects real-time streaming ingestion tasks.
3Submit Supervisor Speccurl -X POST -H 'Content-Type: application/json' -d @spec.json http://localhost:8081/druid/indexer/v1/supervisorProvisions Kafka indexing supervisor.
4Terminate Supervisorcurl -X POST http://localhost:8081/druid/indexer/v1/supervisor/my-datasource/terminateStops streaming ingestion supervisor.
5List Active Taskscurl http://localhost:8081/druid/indexer/v1/tasksInspects background indexing tasks.
6Kill Taskcurl -X POST http://localhost:8081/druid/indexer/v1/task/{taskId}/shutdownTerminates background indexing task.
7Get Cluster Load Statuscurl http://localhost:8081/druid/coordinator/v1/loadstatusEvaluates segment loading progress.
8Get Segments Metadatacurl http://localhost:8081/druid/coordinator/v1/metadata/datasources/telemetry/segmentsInspects time-series data segments.
9Disable Datasource Segmentscurl -X DELETE http://localhost:8081/druid/coordinator/v1/datasources/telemetrySoft-deletes datasource segments.
10Execute SQL Querycurl -X POST -H 'Content-Type: application/json' -d '{"query":"SELECT COUNT(*) FROM telemetry"}' http://localhost:8088/druid/v2/sqlExecutes high-speed analytical query.
11Get Coordinator Serverscurl http://localhost:8081/druid/coordinator/v1/serversInspects historical and middlemanager nodes.
12Set Retention Rulescurl -X POST -H 'Content-Type: application/json' -d '[{"type":"period","period":"P90D","action":"drop"}]' http://localhost:8081/druid/coordinator/v1/rules/telemetryConfigures automated data lifecycle retention.
13Get Retention Rulescurl http://localhost:8081/druid/coordinator/v1/rules/telemetryInspects active data retention rules.
14Get Compaction Statuscurl http://localhost:8081/druid/coordinator/v1/compaction/statusEvaluates historical segment compaction.
15Submit Compaction Configcurl -X POST -H 'Content-Type: application/json' -d '{"dataSource":"telemetry"}' http://localhost:8081/druid/coordinator/v1/compaction/configurationsConfigures background segment compaction.
16Get Overlord Leadercurl http://localhost:8090/druid/indexer/v1/leaderInspects Overlord master leader node.
17Get MiddleManager Slotscurl http://localhost:8090/druid/indexer/v1/workersInspects indexing worker task slots.
18Submit Batch Ingestioncurl -X POST -H 'Content-Type: application/json' -d @batch-spec.json http://localhost:8081/druid/indexer/v1/taskProvisions batch indexing task.
19Get Query Metricscurl http://localhost:8088/druid/v2/metricsPulls broker query latency telemetry.
20Get Datasource Listcurl http://localhost:8081/druid/coordinator/v1/datasourcesLists all active analytical datasources.
21Kill Datasource Permanentlycurl -X DELETE http://localhost:8081/druid/coordinator/v1/datasources/telemetry?intervals=1000/3000Purges historical segment files.
22Get Broker Statuscurl http://localhost:8082/statusInspects query broker service state.
23Get Historical Statuscurl http://localhost:8083/statusInspects storage historical node state.
24Get Indexing Service Configcurl http://localhost:8081/druid/coordinator/v1/configInspects cluster coordinator parameters.
25Update Dynamic Configcurl -X POST -H 'Content-Type: application/json' -d '{"maxSegmentsToMove":10}' http://localhost:8081/druid/coordinator/v1/configModifies dynamic cluster parameters.
26Get Lookups Statecurl http://localhost:8081/druid/coordinator/v1/lookups/configInspects in-memory dimension lookups.
27Update Lookup Configcurl -X POST -H 'Content-Type: application/json' -d @lookup.json http://localhost:8081/druid/coordinator/v1/lookups/config/tier1Provisions dimension lookup table.
28Get Task Reportscurl http://localhost:8081/druid/indexer/v1/task/{taskId}/reportsInspects indexing task execution report.
29Get Metastore Metadata Store Statecurl http://localhost:8081/druid/coordinator/v1/metadata/storagestatsInspects metadata database storage.
30Restart Druid Servicesudo systemctl restart druid-coordinatorRestarts coordinator daemon.
31Suspend Supervisorcurl -X POST http://localhost:8081/druid/indexer/v1/supervisor/my-datasource/suspendPauses real-time ingestion supervisor.
32Resume Supervisorcurl -X POST http://localhost:8081/druid/indexer/v1/supervisor/my-datasource/resumeResumes real-time ingestion supervisor.
33Reset Supervisorcurl -X POST http://localhost:8081/druid/indexer/v1/supervisor/my-datasource/resetResets offsets for streaming supervisor.
34Get Supervisor Statscurl http://localhost:8081/druid/indexer/v1/supervisor/my-datasource/statsPulls real-time processing statistics.
35Get Supervisor Statuscurl http://localhost:8081/druid/indexer/v1/supervisor/my-datasource/statusInspects active lag and task health.
36Get All Supervisors Statuscurl http://localhost:8081/druid/indexer/v1/supervisor?fullPulls detailed status for all supervisors.
37Get Task Statuscurl http://localhost:8081/druid/indexer/v1/task/{taskId}/statusInspects state of specific indexing task.
38Get Task Logcurl http://localhost:8081/druid/indexer/v1/task/{taskId}/logDownloads stdout log of indexing task.
39Get Running Taskscurl http://localhost:8081/druid/indexer/v1/runningTasksLists all active tasks across MiddleManagers.
40Get Pending Taskscurl http://localhost:8081/druid/indexer/v1/pendingTasksLists tasks queued for execution.
41Get Waiting Taskscurl http://localhost:8081/druid/indexer/v1/waitingTasksLists tasks waiting for lock acquisition.
42Get Complete Taskscurl http://localhost:8081/druid/indexer/v1/completeTasksLists finished indexing tasks.
43Get Coordinator Dynamic Configcurl http://localhost:8081/druid/coordinator/v1/configInspects segment balancing parameters.
44Get Historical Tier Segmentscurl http://localhost:8081/druid/coordinator/v1/tiersLists historical tier segment distribution.
45Get Tier Load Statuscurl http://localhost:8081/druid/coordinator/v1/loadqueueInspects segment download queue per node.
46Get Datasource Segment Intervalscurl http://localhost:8081/druid/coordinator/v1/datasources/telemetry/intervalsLists active time intervals in datasource.
47Get Datasource Segment Detailscurl http://localhost:8081/druid/coordinator/v1/datasources/telemetry/intervals/2026-07-01_2026-08-01Inspects segments in specific interval.
48Enable Datasource Segmentcurl -X POST http://localhost:8081/druid/coordinator/v1/datasources/telemetry/segments/{segmentId}Re-enables soft-deleted segment.
49Mark Segment Unusedcurl -X DELETE http://localhost:8081/druid/coordinator/v1/datasources/telemetry/segments/{segmentId}Soft-deletes specific segment ID.
50Get Unused Segmentscurl http://localhost:8081/druid/coordinator/v1/indexing/test/headInspects metadata for unused segments.
51Submit Kill Taskcurl -X POST -H 'Content-Type: application/json' -d '{"type":"kill","dataSource":"telemetry","interval":"2026-01-01/2026-02-01"}' http://localhost:8081/druid/indexer/v1/taskPermanently purges unused segments from deep storage.
52Get Cluster Server Informationcurl http://localhost:8081/druid/coordinator/v1/servers?fullInspects disk usage and tiers per server.
53Get Server Segment Listcurl http://localhost:8081/druid/coordinator/v1/servers/{serverName}/segmentsLists all segments hosted on specific Historical node.
54Get Router Healthcurl http://localhost:8888/status/healthVerifies Router query gateway health.
55Get Broker Cluster Nodescurl http://localhost:8082/druid/v2/datasourcesLists datasources available via Broker.
56Execute Native JSON Querycurl -X POST -H 'Content-Type: application/json' -d @query.json http://localhost:8082/druid/v2Executes low-level JSON aggregation query.
57Cancel Broker SQL Querycurl -X DELETE http://localhost:8082/druid/v2/sql/{queryId}Cancels active SQL query on Broker.
58Get Broker SQL Explaincurl -X POST -H 'Content-Type: application/json' -d '{"query":"EXPLAIN PLAN FOR SELECT COUNT(*) FROM telemetry"}' http://localhost:8082/druid/v2/sqlExplains SQL execution plan.
59Get MiddleManager Workers Listcurl http://localhost:8090/druid/indexer/v1/workersLists MiddleManager instances and capacities.
60Disable MiddleManager Workercurl -X POST http://localhost:8090/druid/indexer/v1/worker/{workerHost}/disableDisables new task assignment to worker.
61Enable MiddleManager Workercurl -X POST http://localhost:8090/druid/indexer/v1/worker/{workerHost}/enableRe-enables task assignment to worker.
62Get Overlord Audit Logscurl http://localhost:8090/druid/indexer/v1/audit/logInspects task submission audit history.
63Get Lookup Tier Nodescurl http://localhost:8081/druid/coordinator/v1/lookups/config/tiersLists lookup configuration tiers.
64Delete Lookup Configcurl -X DELETE http://localhost:8081/druid/coordinator/v1/lookups/config/tier1/my_lookupDeletes dimension lookup table.
65Get Rule Historycurl http://localhost:8081/druid/coordinator/v1/rules/history?count=10Inspects historical retention rule changes.
66Get Dynamic Config Historycurl http://localhost:8081/druid/coordinator/v1/config/history?count=10Inspects historical coordinator config changes.
67Get Compaction Task Configcurl http://localhost:8081/druid/coordinator/v1/compaction/config/telemetryInspects compaction settings for datasource.
68Delete Compaction Configcurl -X DELETE http://localhost:8081/druid/coordinator/v1/compaction/config/telemetryDeletes automated compaction configuration.
69Get Indexing Service Scaling Statscurl http://localhost:8090/druid/indexer/v1/autoscaleInspects MiddleManager auto-scaling state.
70Get Deep Storage Locationcurl http://localhost:8081/druid/coordinator/v1/metadata/datasourcesInspects S3/HDFS deep storage paths.
71Query Druid Information Schema TablesSELECT * FROM INFORMATION_SCHEMA.TABLES WHERE TABLE_SCHEMA='druid';Queries SQL metadata schema for tables.
72Query Druid Information Schema ColumnsSELECT * FROM INFORMATION_SCHEMA.COLUMNS WHERE TABLE_NAME='telemetry';Queries SQL metadata schema for columns.
73Query Druid Information Schema SegmentsSELECT * FROM sys.segments WHERE datasource='telemetry';Queries system table for segment details.
74Query Druid Information Schema ServersSELECT * FROM sys.servers;Queries system table for active nodes.
75Query Druid Information Schema TasksSELECT * FROM sys.tasks WHERE status='RUNNING';Queries system table for active tasks.
76Check Druid Versioncurl http://localhost:8081/statusOutputs Druid release version string.
77Check MiddleManager Healthcurl http://localhost:8091/status/healthVerifies MiddleManager process health.
78Check Historical Node Healthcurl http://localhost:8083/status/healthVerifies Historical storage node health.
79Check Overlord Healthcurl http://localhost:8090/status/healthVerifies Overlord task master health.
80Restart Historical Daemonsudo systemctl restart druid-historicalRestarts Historical node service.
81Restart MiddleManager Daemonsudo systemctl restart druid-middlemanagerRestarts MiddleManager node service.
82Restart Broker Daemonsudo systemctl restart druid-brokerRestarts Broker service.
83Restart Router Daemonsudo systemctl restart druid-routerRestarts Router gateway service.
84Restart Overlord Daemonsudo systemctl restart druid-overlordRestarts Overlord task master service.
85Flush Segment CachePOST http://localhost:8083/druid/historical/v1/cache/flushFlushes Historical node segment cache.
86Set Max Query Timeout Sessioncurl -X POST -H 'Content-Type: application/json' -d '{"query":"SELECT COUNT(*) FROM telemetry","context":{"timeout":30000}}' http://localhost:8088/druid/v2/sqlCaps query execution to 30 seconds.
87Set Query Timezone Sessioncurl -X POST -H 'Content-Type: application/json' -d '{"query":"SELECT COUNT(*) FROM telemetry","context":{"sqlTimeZone":"UTC"}}' http://localhost:8088/druid/v2/sqlSets SQL session timezone to UTC.
88Set Vector Execution Strategycurl -X POST -H 'Content-Type: application/json' -d '{"query":"SELECT COUNT(*) FROM telemetry","context":{"vectorize":"true"}}' http://localhost:8088/druid/v2/sqlEnforces vectorized query execution engine.
89Check Task Slot UtilizationSELECT sum(worker_slots) FROM sys.workers;Calculates total indexing slots across cluster.
90Check Segment Size DistributionSELECT datasource, AVG(size) FROM sys.segments GROUP BY datasource;Calculates average segment file size in bytes.
91Check Deep Storage Sync Statuscurl http://localhost:8081/druid/coordinator/v1/metadata/statusVerifies metadata database sync with deep storage.
92Purge All Audit HistoryPOST http://localhost:8081/druid/coordinator/v1/history/purgeClears administrative audit log history.
93Set Router Balancing Strategycurl -X POST -H 'Content-Type: application/json' -d '{"strategy":"connectionCount"}' http://localhost:8888/druid/router/v1/configUpdates Router broker balancing strategy.
94Check JVM Memory Usagecurl http://localhost:8081/status/propertiesInspects JVM system properties and memory limits.
95Verify Druid API Versioncurl http://localhost:8081/status/versionOutputs API version string.
96Check Coordinator Thread Poolcurl http://localhost:8081/druid/coordinator/v1/metricsInspects coordinator thread pool latency.
97Check Indexing Task Queue Sizecurl http://localhost:8090/druid/indexer/v1/queue/statusInspects pending task queue count.
98Force Segment LoadPOST http://localhost:8081/druid/coordinator/v1/rules/loadNowForces immediate loading of offline segments.
99Force Segment UnloadPOST http://localhost:8081/druid/coordinator/v1/rules/dropNowForces immediate unloading of active segments.
100Check Cluster Status Summarycurl http://localhost:8081/druid/coordinator/v1/cluster/statusOutputs high-level cluster health summary.
101Check Coordinator Health HTTPcurl http://localhost:8081/status/healthQueries Coordinator health status.
102Check Overlord Health HTTPcurl http://localhost:8090/status/healthQueries Overlord health status.
103Check Broker Health HTTPcurl http://localhost:8082/status/healthQueries Broker health status.
104Check Historical Health HTTPcurl http://localhost:8083/status/healthQueries Historical health status.
105Check Router Health HTTPcurl http://localhost:8888/status/healthQueries Router health status.
106Check MiddleManager Health HTTPcurl http://localhost:8091/status/healthQueries MiddleManager health status.
107Check Coordinator Status HTTPcurl http://localhost:8081/statusOutputs Coordinator process info and uptime.
108Check Overlord Status HTTPcurl http://localhost:8090/statusOutputs Overlord process info and uptime.
109Check Broker Status HTTPcurl http://localhost:8082/statusOutputs Broker process info and uptime.
110Check Historical Status HTTPcurl http://localhost:8083/statusOutputs Historical process info and uptime.
111Check Router Status HTTPcurl http://localhost:8888/statusOutputs Router process info and uptime.
112Check MiddleManager Status HTTPcurl http://localhost:8091/statusOutputs MiddleManager process info and uptime.
113Check Coordinator Versioncurl http://localhost:8081/status/versionOutputs Coordinator release version string.
114Check Overlord Versioncurl http://localhost:8090/status/versionOutputs Overlord release version string.
115Check Broker Versioncurl http://localhost:8082/status/versionOutputs Broker release version string.
116Check Historical Versioncurl http://localhost:8083/status/versionOutputs Historical release version string.
117Check Router Versioncurl http://localhost:8888/status/versionOutputs Router release version string.
118Check MiddleManager Versioncurl http://localhost:8091/status/versionOutputs MiddleManager release version string.
119Check Active Supervisors Countcurl -s http://localhost:8081/druid/indexer/v1/supervisor | grep -o '"' | wc -lCounts active streaming supervisors.
120Check Active Tasks Countcurl -s http://localhost:8081/druid/indexer/v1/tasks | grep -o '"id"' | wc -lCounts active indexing tasks.
121Check Active Datasources Countcurl -s http://localhost:8081/druid/coordinator/v1/datasources | grep -o '"' | wc -lCounts active datasources.
122Check Active Historical Servers Countcurl -s http://localhost:8081/druid/coordinator/v1/servers | grep -o '"' | wc -lCounts connected Historical servers.
123Check SQL Query Pingcurl -X POST -H 'Content-Type: application/json' -d '{"query":"SELECT 1"}' http://localhost:8088/druid/v2/sqlExecutes simple SQL ping test.
124Check SQL Query Timezonecurl -X POST -H 'Content-Type: application/json' -d '{"query":"SELECT CURRENT_TIMESTAMP"}' http://localhost:8088/druid/v2/sqlQueries active server timestamp.
125Check SQL Query Versioncurl -X POST -H 'Content-Type: application/json' -d '{"query":"SELECT CURRENT_USER"}' http://localhost:8088/druid/v2/sqlQueries current execution user.
126Check System Tasks Count SQLcurl -X POST -H 'Content-Type: application/json' -d '{"query":"SELECT count(*) FROM sys.tasks"}' http://localhost:8088/druid/v2/sqlCounts total tasks via SQL sys schema.
127Check System Segments Count SQLcurl -X POST -H 'Content-Type: application/json' -d '{"query":"SELECT count(*) FROM sys.segments"}' http://localhost:8088/druid/v2/sqlCounts total segments via SQL sys schema.
128Check System Servers Count SQLcurl -X POST -H 'Content-Type: application/json' -d '{"query":"SELECT count(*) FROM sys.servers"}' http://localhost:8088/druid/v2/sqlCounts total servers via SQL sys schema.
129Check System Server Segments Count SQLcurl -X POST -H 'Content-Type: application/json' -d '{"query":"SELECT count(*) FROM sys.server_segments"}' http://localhost:8088/druid/v2/sqlCounts total server segment mappings.
130Check System Supervisors Count SQLcurl -X POST -H 'Content-Type: application/json' -d '{"query":"SELECT count(*) FROM sys.supervisors"}' http://localhost:8088/druid/v2/sqlCounts total supervisors via SQL sys schema.
131Check Cluster Health (Alt 130)curl http://localhost:8081/status/healthVerifies coordinator node health.
132List Supervisors (Alt 131)curl http://localhost:8081/druid/indexer/v1/supervisorInspects real-time streaming ingestion tasks.
133Submit Supervisor Spec (Alt 132)curl -X POST -H 'Content-Type: application/json' -d @spec.json http://localhost:8081/druid/indexer/v1/supervisorProvisions Kafka indexing supervisor.
134Terminate Supervisor (Alt 133)curl -X POST http://localhost:8081/druid/indexer/v1/supervisor/my-datasource/terminateStops streaming ingestion supervisor.
135List Active Tasks (Alt 134)curl http://localhost:8081/druid/indexer/v1/tasksInspects background indexing tasks.
136Kill Task (Alt 135)curl -X POST http://localhost:8081/druid/indexer/v1/task/{taskId}/shutdownTerminates background indexing task.
137Get Cluster Load Status (Alt 136)curl http://localhost:8081/druid/coordinator/v1/loadstatusEvaluates segment loading progress.
138Get Segments Metadata (Alt 137)curl http://localhost:8081/druid/coordinator/v1/metadata/datasources/telemetry/segmentsInspects time-series data segments.
139Disable Datasource Segments (Alt 138)curl -X DELETE http://localhost:8081/druid/coordinator/v1/datasources/telemetrySoft-deletes datasource segments.
140Execute SQL Query (Alt 139)curl -X POST -H 'Content-Type: application/json' -d '{"query":"SELECT COUNT(*) FROM telemetry"}' http://localhost:8088/druid/v2/sqlExecutes high-speed analytical query.
141Get Coordinator Servers (Alt 140)curl http://localhost:8081/druid/coordinator/v1/serversInspects historical and middlemanager nodes.
142Set Retention Rules (Alt 141)curl -X POST -H 'Content-Type: application/json' -d '[{"type":"period","period":"P90D","action":"drop"}]' http://localhost:8081/druid/coordinator/v1/rules/telemetryConfigures automated data lifecycle retention.
143Get Retention Rules (Alt 142)curl http://localhost:8081/druid/coordinator/v1/rules/telemetryInspects active data retention rules.
144Get Compaction Status (Alt 143)curl http://localhost:8081/druid/coordinator/v1/compaction/statusEvaluates historical segment compaction.
145Submit Compaction Config (Alt 144)curl -X POST -H 'Content-Type: application/json' -d '{"dataSource":"telemetry"}' http://localhost:8081/druid/coordinator/v1/compaction/configurationsConfigures background segment compaction.
146Get Overlord Leader (Alt 145)curl http://localhost:8090/druid/indexer/v1/leaderInspects Overlord master leader node.
147Get MiddleManager Slots (Alt 146)curl http://localhost:8090/druid/indexer/v1/workersInspects indexing worker task slots.
148Submit Batch Ingestion (Alt 147)curl -X POST -H 'Content-Type: application/json' -d @batch-spec.json http://localhost:8081/druid/indexer/v1/taskProvisions batch indexing task.
149Get Query Metrics (Alt 148)curl http://localhost:8088/druid/v2/metricsPulls broker query latency telemetry.
150Get Datasource List (Alt 149)curl http://localhost:8081/druid/coordinator/v1/datasourcesLists all active analytical datasources.
151Kill Datasource Permanently (Alt 150)curl -X DELETE http://localhost:8081/druid/coordinator/v1/datasources/telemetry?intervals=1000/3000Purges historical segment files.
152Get Broker Status (Alt 151)curl http://localhost:8082/statusInspects query broker service state.
153Get Historical Status (Alt 152)curl http://localhost:8083/statusInspects storage historical node state.
154Get Indexing Service Config (Alt 153)curl http://localhost:8081/druid/coordinator/v1/configInspects cluster coordinator parameters.
155Update Dynamic Config (Alt 154)curl -X POST -H 'Content-Type: application/json' -d '{"maxSegmentsToMove":10}' http://localhost:8081/druid/coordinator/v1/configModifies dynamic cluster parameters.
156Get Lookups State (Alt 155)curl http://localhost:8081/druid/coordinator/v1/lookups/configInspects in-memory dimension lookups.
157Update Lookup Config (Alt 156)curl -X POST -H 'Content-Type: application/json' -d @lookup.json http://localhost:8081/druid/coordinator/v1/lookups/config/tier1Provisions dimension lookup table.
158Get Task Reports (Alt 157)curl http://localhost:8081/druid/indexer/v1/task/{taskId}/reportsInspects indexing task execution report.
159Get Metastore Metadata Store State (Alt 158)curl http://localhost:8081/druid/coordinator/v1/metadata/storagestatsInspects metadata database storage.
160Restart Druid Service (Alt 159)sudo systemctl restart druid-coordinatorRestarts coordinator daemon.
161Suspend Supervisor (Alt 160)curl -X POST http://localhost:8081/druid/indexer/v1/supervisor/my-datasource/suspendPauses real-time ingestion supervisor.
162Resume Supervisor (Alt 161)curl -X POST http://localhost:8081/druid/indexer/v1/supervisor/my-datasource/resumeResumes real-time ingestion supervisor.
163Reset Supervisor (Alt 162)curl -X POST http://localhost:8081/druid/indexer/v1/supervisor/my-datasource/resetResets offsets for streaming supervisor.
164Get Supervisor Stats (Alt 163)curl http://localhost:8081/druid/indexer/v1/supervisor/my-datasource/statsPulls real-time processing statistics.
165Get Supervisor Status (Alt 164)curl http://localhost:8081/druid/indexer/v1/supervisor/my-datasource/statusInspects active lag and task health.
166Get All Supervisors Status (Alt 165)curl http://localhost:8081/druid/indexer/v1/supervisor?fullPulls detailed status for all supervisors.
167Get Task Status (Alt 166)curl http://localhost:8081/druid/indexer/v1/task/{taskId}/statusInspects state of specific indexing task.
168Get Task Log (Alt 167)curl http://localhost:8081/druid/indexer/v1/task/{taskId}/logDownloads stdout log of indexing task.
169Get Running Tasks (Alt 168)curl http://localhost:8081/druid/indexer/v1/runningTasksLists all active tasks across MiddleManagers.
170Get Pending Tasks (Alt 169)curl http://localhost:8081/druid/indexer/v1/pendingTasksLists tasks queued for execution.
171Get Waiting Tasks (Alt 170)curl http://localhost:8081/druid/indexer/v1/waitingTasksLists tasks waiting for lock acquisition.
172Get Complete Tasks (Alt 171)curl http://localhost:8081/druid/indexer/v1/completeTasksLists finished indexing tasks.
173Get Coordinator Dynamic Config (Alt 172)curl http://localhost:8081/druid/coordinator/v1/configInspects segment balancing parameters.
174Get Historical Tier Segments (Alt 173)curl http://localhost:8081/druid/coordinator/v1/tiersLists historical tier segment distribution.
175Get Tier Load Status (Alt 174)curl http://localhost:8081/druid/coordinator/v1/loadqueueInspects segment download queue per node.
176Get Datasource Segment Intervals (Alt 175)curl http://localhost:8081/druid/coordinator/v1/datasources/telemetry/intervalsLists active time intervals in datasource.
177Get Datasource Segment Details (Alt 176)curl http://localhost:8081/druid/coordinator/v1/datasources/telemetry/intervals/2026-07-01_2026-08-01Inspects segments in specific interval.
178Enable Datasource Segment (Alt 177)curl -X POST http://localhost:8081/druid/coordinator/v1/datasources/telemetry/segments/{segmentId}Re-enables soft-deleted segment.
179Mark Segment Unused (Alt 178)curl -X DELETE http://localhost:8081/druid/coordinator/v1/datasources/telemetry/segments/{segmentId}Soft-deletes specific segment ID.
180Get Unused Segments (Alt 179)curl http://localhost:8081/druid/coordinator/v1/indexing/test/headInspects metadata for unused segments.
181Submit Kill Task (Alt 180)curl -X POST -H 'Content-Type: application/json' -d '{"type":"kill","dataSource":"telemetry","interval":"2026-01-01/2026-02-01"}' http://localhost:8081/druid/indexer/v1/taskPermanently purges unused segments from deep storage.
182Get Cluster Server Information (Alt 181)curl http://localhost:8081/druid/coordinator/v1/servers?fullInspects disk usage and tiers per server.
183Get Server Segment List (Alt 182)curl http://localhost:8081/druid/coordinator/v1/servers/{serverName}/segmentsLists all segments hosted on specific Historical node.
184Get Router Health (Alt 183)curl http://localhost:8888/status/healthVerifies Router query gateway health.
185Get Broker Cluster Nodes (Alt 184)curl http://localhost:8082/druid/v2/datasourcesLists datasources available via Broker.
186Execute Native JSON Query (Alt 185)curl -X POST -H 'Content-Type: application/json' -d @query.json http://localhost:8082/druid/v2Executes low-level JSON aggregation query.
187Cancel Broker SQL Query (Alt 186)curl -X DELETE http://localhost:8082/druid/v2/sql/{queryId}Cancels active SQL query on Broker.
188Get Broker SQL Explain (Alt 187)curl -X POST -H 'Content-Type: application/json' -d '{"query":"EXPLAIN PLAN FOR SELECT COUNT(*) FROM telemetry"}' http://localhost:8082/druid/v2/sqlExplains SQL execution plan.
189Get MiddleManager Workers List (Alt 188)curl http://localhost:8090/druid/indexer/v1/workersLists MiddleManager instances and capacities.
190Disable MiddleManager Worker (Alt 189)curl -X POST http://localhost:8090/druid/indexer/v1/worker/{workerHost}/disableDisables new task assignment to worker.
191Enable MiddleManager Worker (Alt 190)curl -X POST http://localhost:8090/druid/indexer/v1/worker/{workerHost}/enableRe-enables task assignment to worker.
192Get Overlord Audit Logs (Alt 191)curl http://localhost:8090/druid/indexer/v1/audit/logInspects task submission audit history.
193Get Lookup Tier Nodes (Alt 192)curl http://localhost:8081/druid/coordinator/v1/lookups/config/tiersLists lookup configuration tiers.
194Delete Lookup Config (Alt 193)curl -X DELETE http://localhost:8081/druid/coordinator/v1/lookups/config/tier1/my_lookupDeletes dimension lookup table.
195Get Rule History (Alt 194)curl http://localhost:8081/druid/coordinator/v1/rules/history?count=10Inspects historical retention rule changes.
196Get Dynamic Config History (Alt 195)curl http://localhost:8081/druid/coordinator/v1/config/history?count=10Inspects historical coordinator config changes.
197Get Compaction Task Config (Alt 196)curl http://localhost:8081/druid/coordinator/v1/compaction/config/telemetryInspects compaction settings for datasource.
198Delete Compaction Config (Alt 197)curl -X DELETE http://localhost:8081/druid/coordinator/v1/compaction/config/telemetryDeletes automated compaction configuration.
199Get Indexing Service Scaling Stats (Alt 198)curl http://localhost:8090/druid/indexer/v1/autoscaleInspects MiddleManager auto-scaling state.
200Get Deep Storage Location (Alt 199)curl http://localhost:8081/druid/coordinator/v1/metadata/datasourcesInspects S3/HDFS deep storage paths.

DuckDB OLAP

Technical Architecture & Overview

DuckDB is an embedded, in-process columnar analytical database engine optimized for zero-copy vectorized OLAP queries. It runs inside the host process (Python, R, C++, Java), directly querying Parquet, CSV, JSON, Iceberg, and Delta Lake files without requiring a separate database server daemon.

Primary Use Cases: Embedded local analytics, Python/Pandas/Polars data science pipelines, serverless cloud file querying (MotherDuck), fast ETL file transformations, and interactive notebook analytics.

Core Components: In-Process Vectorized Execution Engine, Hyper-Pipeline Parallel Scheduler, Columnar Storage Engine, Extension Manager, and HTTPFS Cloud Reader.

Exhaustive 200-command administrative, diagnostic, and operational matrix for DuckDB.

#Operation / ActionCLI / SQL SyntaxDescription
1Open Interactive CLIduckdb enterprise.dbLaunches interactive analytical shell.
2Export Database BackupEXPORT DATABASE 'backup_dir';Exports schema and tables to files.
3Import DatabaseIMPORT DATABASE 'backup_dir';Restores database from files.
4Install ExtensionINSTALL httpfs;Provisions cloud storage extension.
5Load ExtensionLOAD httpfs;Activates cloud storage extension.
6Query Parquet File DirectlySELECT * FROM read_parquet('s3://bucket/data.parquet');Queries cloud file without loading.
7Query CSV File DirectlySELECT * FROM read_csv_auto('data.csv');Queries CSV file directly.
8Create Table from ParquetCREATE TABLE users AS SELECT * FROM 'data.parquet';Ingests file into persistent table.
9Explain Query ExecutionEXPLAIN SELECT * FROM users;Evaluates vectorized query plan.
10Check Memory LimitPRAGMA memory_limit='8GB';Sets maximum RAM allocation.
11Check Thread CountPRAGMA threads=8;Configures parallel execution threads.
12Check Table SchemaDESCRIBE users;Inspects table column data types.
13List TablesSHOW TABLES;Outputs all database tables.
14Attach External DatabaseATTACH 'postgres:dbname=enterprise user=postgres' AS pg_db (TYPE POSTGRES);Mounts external PostgreSQL database.
15Detach External DatabaseDETACH pg_db;Unmounts external database.
16Create Secret (AWS S3)CREATE SECRET (TYPE S3, KEY_ID 'AKIA...', SECRET 'secret...');Stores cloud credentials securely.
17Query Information SchemaSELECT * FROM information_schema.tables;Inspects metadata catalogue.
18Check Database Storage SizeSELECT * FROM pragma_database_size();Calculates database file disk size.
19Vacuum DefragmentVACUUM;Rebuilds database file to reclaim space.
20Checkpoint FileCHECKPOINT;Forces immediate WAL flush to disk.
21Create ViewCREATE VIEW user_summary AS SELECT * FROM users WHERE active = true;Provisions virtual query view.
22Create Macro (Function)CREATE MACRO add_tax(x) AS x * 1.08;Provisions reusable query macro.
23Export to ParquetCOPY users TO 'users.parquet' (FORMAT PARQUET);Exports table to compressed Parquet.
24Export to CSVCOPY users TO 'users.csv' (HEADER, DELIMITER ',');Exports table to CSV file.
25Check DuckDB VersionSELECT version();Outputs running DuckDB binary version.
26Install MotherDuck ExtensionINSTALL motherduck; LOAD motherduck;Provisions cloud-native DuckDB extension.
27Connect MotherDuck DatabaseATTACH 'md:my_db' AS cloud_db;Connects to cloud MotherDuck database.
28Check Active ExtensionsSELECT * FROM duckdb_extensions();Inspects loaded software modules.
29Analyze Query ProfilePRAGMA enable_profiling; SELECT * FROM users; PRAGMA profiling_output='profile.json';Generates deep performance profile.
30Exit CLI.exitCloses interactive shell.
31Install Spatial ExtensionINSTALL spatial; LOAD spatial;Provisions GIS spatial query extension.
32Install JSON ExtensionINSTALL json; LOAD json;Provisions high-speed JSON extraction extension.
33Install Iceberg ExtensionINSTALL iceberg; LOAD iceberg;Provisions Apache Iceberg lakehouse extension.
34Install Delta ExtensionINSTALL delta; LOAD delta;Provisions Delta Lake table reader extension.
35Install MySQL ExtensionINSTALL mysql; LOAD mysql;Provisions direct MySQL database connector.
36Install SQLite ExtensionINSTALL sqlite; LOAD sqlite;Provisions direct SQLite file connector.
37Install Arrow ExtensionINSTALL arrow; LOAD arrow;Provisions Apache Arrow in-memory zero-copy extension.
38Attach SQLite DatabaseATTACH 'archive.db' AS sqlite_db (TYPE SQLITE);Mounts SQLite file as database schema.
39Attach MySQL DatabaseATTACH 'host=10.0.0.1 user=root dbname=prod' AS mysql_db (TYPE MYSQL);Mounts remote MySQL database.
40Query Iceberg Table DirectlySELECT * FROM iceberg_scan('s3://bucket/iceberg_table');Queries Iceberg lakehouse table directly.
41Query Delta Table DirectlySELECT * FROM delta_scan('s3://bucket/delta_table');Queries Delta Lake table directly.
42Query JSON File DirectlySELECT * FROM read_json_auto('data.json');Queries JSON file directly with schema inference.
43Query NDJSON File DirectlySELECT * FROM read_ndjson_objects('logs.json');Queries newline-delimited JSON objects.
44Query Compressed Gzip CSVSELECT * FROM read_csv_auto('data.csv.gz');Queries gzip-compressed CSV directly.
45Query Compressed Zstd ParquetSELECT * FROM read_parquet('data.parquet.zst');Queries zstd-compressed Parquet directly.
46Query Glob Parquet FilesSELECT * FROM read_parquet('s3://bucket/year=2026/*/*.parquet');Queries multiple Parquet files via glob pattern.
47Copy CSV with CompressionCOPY users TO 'data.csv.gz' (HEADER, COMPRESSION GZIP);Exports table to gzip-compressed CSV.
48Copy Parquet with Zstd CompressionCOPY users TO 'data.parquet' (FORMAT PARQUET, COMPRESSION ZSTD);Exports table to Zstd-compressed Parquet.
49Copy Parquet with Snappy CompressionCOPY users TO 'data.parquet' (FORMAT PARQUET, COMPRESSION SNAPPY);Exports table to Snappy-compressed Parquet.
50Copy JSON to FileCOPY users TO 'users.json' (ARRAY true);Exports table to JSON file.
51Create Table As SelectCREATE TABLE analytics_users AS SELECT id, name FROM read_csv_auto('users.csv');Ingests CSV into new persistent table.
52Create Temp TableCREATE TEMP TABLE tmp_users AS SELECT * FROM users;Provisions in-memory temporary table.
53Create Table PartitionedCOPY users TO 'output_dir' (FORMAT PARQUET, PARTITION_BY (dept));Exports table partitioned into subdirectories.
54Create Table OverwriteCREATE OR REPLACE TABLE users AS SELECT * FROM 'users.parquet';Replaces existing table with new schema/data.
55Create SequenceCREATE SEQUENCE user_seq START 1;Provisions sequential ID generator.
56Get Sequence Next ValueSELECT nextval('user_seq');Fetches next value from sequence.
57Drop SequenceDROP SEQUENCE user_seq;Deletes sequence object.
58Create Table Unique ConstraintCREATE TABLE users (id INT UNIQUE, email VARCHAR);Provisions unique constraint on column.
59Create Table Primary KeyCREATE TABLE users (id INT PRIMARY KEY, name VARCHAR);Provisions primary key constraint on column.
60Create Table Check ConstraintCREATE TABLE users (id INT, age INT CHECK (age >= 18));Provisions check constraint on column.
61Alter Table Add ColumnALTER TABLE users ADD COLUMN age INT DEFAULT 0;Appends column to existing table.
62Alter Table Drop ColumnALTER TABLE users DROP COLUMN age;Removes column from existing table.
63Alter Table Rename ColumnALTER TABLE users RENAME COLUMN name TO full_name;Renames column in table.
64Alter Table Rename TableALTER TABLE users RENAME TO app_users;Renames table in schema.
65Alter Table Column TypeALTER TABLE users ALTER COLUMN id TYPE BIGINT;Changes column data type definition.
66Create IndexCREATE INDEX idx_user_email ON users(email);Provisions secondary index file.
67Drop IndexDROP INDEX idx_user_email;Removes secondary index file.
68Show IndexesSELECT * FROM duckdb_indexes();Lists all active secondary indexes.
69Set Preserve Insertion OrderPRAGMA preserve_insertion_order = false;Disables order preservation for faster parallel loads.
70Set Default Null OrderPRAGMA default_null_order = 'NULLS LAST';Sets default NULL sorting behavior.
71Set Max Temp DirectoryPRAGMA temp_directory = '/tmp/duckdb_spill';Sets disk directory for query memory spilling.
72Check Disk Spilling MemoryPRAGMA max_temp_directory_size = '100GB';Caps disk spilling storage limit to 100GB.
73Check Memory Usage DetailsSELECT * FROM duckdb_memory();Inspects active RAM allocation per table/buffer.
74Check Query Profiling OutputPRAGMA profiling_mode = 'detailed';Sets detailed execution profiling mode.
75Disable Query ProfilingPRAGMA disable_profiling;Turns off performance profiling.
76Explain Analyze QueryEXPLAIN ANALYZE SELECT count(*) FROM users;Runs query and outputs executed physical operator tree.
77Check Table Column TypesSELECT column_name, data_type FROM information_schema.columns WHERE table_name='users';Inspects schema column types.
78Check Database Catalog ListSELECT * FROM duckdb_databases();Lists all attached databases and types.
79Check Database Schemas ListSELECT * FROM duckdb_schemas();Lists all schemas across attached databases.
80Check Database Tables ListSELECT * FROM duckdb_tables();Lists all tables across attached databases.
81Check Database Views ListSELECT * FROM duckdb_views();Lists all views across attached databases.
82Check Database Settings ListSELECT * FROM duckdb_settings();Lists all configurable system settings.
83Check Database Constraints ListSELECT * FROM duckdb_constraints();Lists all primary, unique, and check constraints.
84Check Database Secrets ListSELECT * FROM duckdb_secrets();Lists stored cloud access credentials.
85Drop SecretDROP SECRET IF EXISTS my_s3_secret;Deletes stored cloud access credential.
86Create Table MacroCREATE MACRO get_users(dept_name) AS TABLE SELECT * FROM users WHERE dept = dept_name;Provisions parameterized table macro.
87Call Table MacroSELECT * FROM get_users('Engineering');Queries table macro.
88Drop MacroDROP MACRO add_tax;Deletes macro function.
89Set Custom User AgentSET custom_user_agent = 'DuckDB-Analytics-Pipeline';Configures HTTP header for cloud requests.
90Set S3 RegionSET s3_region = 'us-east-1';Configures AWS S3 cloud region.
91Set S3 EndpointSET s3_endpoint = 's3.us-east-1.amazonaws.com';Configures custom S3 endpoint URL.
92Set S3 Use SSLSET s3_use_ssl = true;Enforces HTTPS encryption for S3 requests.
93Set S3 Url StyleSET s3_url_style = 'path';Configures path-style S3 URL addressing.
94Check MotherDuck DatabasesSHOW DATABASES;Lists local and MotherDuck cloud databases.
95Create MotherDuck ShareCREATE SHARE my_share FROM my_db (READ_ONLY);Shares MotherDuck cloud database.
96Query MotherDuck System LogsSELECT * FROM motherduck_logs();Queries cloud execution logs.
97Checkpoint Database ManualCHECKPOINT;Forces immediate WAL flush and consolidation.
98Check DuckDB Internal VersionPRAGMA version;Outputs exact DuckDB build commit and version string.
99Set Output Mode Box.mode boxFormats CLI query output as boxed Unicode table.
100Exit DuckDB Shell.quitCloses interactive shell session.
101Check Active Memory Usage BytesSELECT * FROM duckdb_memory();Inspects physical RAM allocation per component.
102Check Max Memory Limit SettingPRAGMA memory_limit;Inspects active RAM limit parameter.
103Set Max Memory Limit 16GBPRAGMA memory_limit='16GB';Configures 16GB max RAM limit.
104Check Active Threads SettingPRAGMA threads;Inspects parallel worker thread count.
105Set Active Threads 16PRAGMA threads=16;Configures 16 parallel worker threads.
106Check Temp Directory PathPRAGMA temp_directory;Inspects disk path for query spilling.
107Set Temp Directory PathPRAGMA temp_directory='/mnt/ssd/duckdb_spill';Sets SSD disk path for query spilling.
108Check Preserve Insertion OrderPRAGMA preserve_insertion_order;Inspects row insertion order preservation setting.
109Check Default Null OrderPRAGMA default_null_order;Inspects NULL sorting behavior (NULLS FIRST vs NULLS LAST).
110Check Default OrderPRAGMA default_order;Inspects default ORDER BY direction (ASC vs DESC).
111Check Enable ProfilingPRAGMA enable_profiling;Activates query performance profiling.
112Check Disable ProfilingPRAGMA disable_profiling;Deactivates query performance profiling.
113Check Profiling ModePRAGMA profiling_mode;Inspects active profiling mode (standard vs detailed).
114Check Profiling Output FilePRAGMA profiling_output;Inspects file path for profiling JSON output.
115Check DuckDB Version String SQLSELECT version();Outputs running DuckDB release version string.
116Check DuckDB Source ID SQLSELECT source_id FROM pragma_version();Outputs git commit hash used to build DuckDB.
117Check Installed Extensions CountSELECT count(*) FROM duckdb_extensions() WHERE installed=true;Counts installed extensions.
118Check Loaded Extensions CountSELECT count(*) FROM duckdb_extensions() WHERE loaded=true;Counts active loaded extensions.
119Check Attached Databases CountSELECT count(*) FROM duckdb_databases();Counts attached databases.
120Check Attached Database NamesSELECT database_name FROM duckdb_databases();Lists names of attached databases.
121Check Schemas ListSELECT schema_name FROM duckdb_schemas();Lists all schemas across attached databases.
122Check Tables CountSELECT count(*) FROM duckdb_tables();Counts total tables in database.
123Check Views CountSELECT count(*) FROM duckdb_views();Counts total views in database.
124Check Indexes CountSELECT count(*) FROM duckdb_indexes();Counts total secondary indexes in database.
125Check Constraints CountSELECT count(*) FROM duckdb_constraints();Counts total table constraints in database.
126Check Secrets CountSELECT count(*) FROM duckdb_secrets();Counts stored cloud access credentials.
127Check System Settings CountSELECT count(*) FROM duckdb_settings();Counts configurable system parameters.
128Check Query Ping SQLSELECT 1;Simple connection ping query returning 1.
129Set CLI Mode Box.mode boxFormats CLI output as Unicode box table.
130Exit DuckDB Shell CLI.quitCloses command-line interactive shell.
131Open Interactive CLI (Alt 130)duckdb enterprise.dbLaunches interactive analytical shell.
132Export Database Backup (Alt 131)EXPORT DATABASE 'backup_dir';Exports schema and tables to files.
133Import Database (Alt 132)IMPORT DATABASE 'backup_dir';Restores database from files.
134Install Extension (Alt 133)INSTALL httpfs;Provisions cloud storage extension.
135Load Extension (Alt 134)LOAD httpfs;Activates cloud storage extension.
136Query Parquet File Directly (Alt 135)SELECT * FROM read_parquet('s3://bucket/data.parquet');Queries cloud file without loading.
137Query CSV File Directly (Alt 136)SELECT * FROM read_csv_auto('data.csv');Queries CSV file directly.
138Create Table from Parquet (Alt 137)CREATE TABLE users AS SELECT * FROM 'data.parquet';Ingests file into persistent table.
139Explain Query Execution (Alt 138)EXPLAIN SELECT * FROM users;Evaluates vectorized query plan.
140Check Memory Limit (Alt 139)PRAGMA memory_limit='8GB';Sets maximum RAM allocation.
141Check Thread Count (Alt 140)PRAGMA threads=8;Configures parallel execution threads.
142Check Table Schema (Alt 141)DESCRIBE users;Inspects table column data types.
143List Tables (Alt 142)SHOW TABLES;Outputs all database tables.
144Attach External Database (Alt 143)ATTACH 'postgres:dbname=enterprise user=postgres' AS pg_db (TYPE POSTGRES);Mounts external PostgreSQL database.
145Detach External Database (Alt 144)DETACH pg_db;Unmounts external database.
146Create Secret (AWS S3) (Alt 145)CREATE SECRET (TYPE S3, KEY_ID 'AKIA...', SECRET 'secret...');Stores cloud credentials securely.
147Query Information Schema (Alt 146)SELECT * FROM information_schema.tables;Inspects metadata catalogue.
148Check Database Storage Size (Alt 147)SELECT * FROM pragma_database_size();Calculates database file disk size.
149Vacuum Defragment (Alt 148)VACUUM;Rebuilds database file to reclaim space.
150Checkpoint File (Alt 149)CHECKPOINT;Forces immediate WAL flush to disk.
151Create View (Alt 150)CREATE VIEW user_summary AS SELECT * FROM users WHERE active = true;Provisions virtual query view.
152Create Macro (Function) (Alt 151)CREATE MACRO add_tax(x) AS x * 1.08;Provisions reusable query macro.
153Export to Parquet (Alt 152)COPY users TO 'users.parquet' (FORMAT PARQUET);Exports table to compressed Parquet.
154Export to CSV (Alt 153)COPY users TO 'users.csv' (HEADER, DELIMITER ',');Exports table to CSV file.
155Check DuckDB Version (Alt 154)SELECT version();Outputs running DuckDB binary version.
156Install MotherDuck Extension (Alt 155)INSTALL motherduck; LOAD motherduck;Provisions cloud-native DuckDB extension.
157Connect MotherDuck Database (Alt 156)ATTACH 'md:my_db' AS cloud_db;Connects to cloud MotherDuck database.
158Check Active Extensions (Alt 157)SELECT * FROM duckdb_extensions();Inspects loaded software modules.
159Analyze Query Profile (Alt 158)PRAGMA enable_profiling; SELECT * FROM users; PRAGMA profiling_output='profile.json';Generates deep performance profile.
160Exit CLI (Alt 159).exitCloses interactive shell.
161Install Spatial Extension (Alt 160)INSTALL spatial; LOAD spatial;Provisions GIS spatial query extension.
162Install JSON Extension (Alt 161)INSTALL json; LOAD json;Provisions high-speed JSON extraction extension.
163Install Iceberg Extension (Alt 162)INSTALL iceberg; LOAD iceberg;Provisions Apache Iceberg lakehouse extension.
164Install Delta Extension (Alt 163)INSTALL delta; LOAD delta;Provisions Delta Lake table reader extension.
165Install MySQL Extension (Alt 164)INSTALL mysql; LOAD mysql;Provisions direct MySQL database connector.
166Install SQLite Extension (Alt 165)INSTALL sqlite; LOAD sqlite;Provisions direct SQLite file connector.
167Install Arrow Extension (Alt 166)INSTALL arrow; LOAD arrow;Provisions Apache Arrow in-memory zero-copy extension.
168Attach SQLite Database (Alt 167)ATTACH 'archive.db' AS sqlite_db (TYPE SQLITE);Mounts SQLite file as database schema.
169Attach MySQL Database (Alt 168)ATTACH 'host=10.0.0.1 user=root dbname=prod' AS mysql_db (TYPE MYSQL);Mounts remote MySQL database.
170Query Iceberg Table Directly (Alt 169)SELECT * FROM iceberg_scan('s3://bucket/iceberg_table');Queries Iceberg lakehouse table directly.
171Query Delta Table Directly (Alt 170)SELECT * FROM delta_scan('s3://bucket/delta_table');Queries Delta Lake table directly.
172Query JSON File Directly (Alt 171)SELECT * FROM read_json_auto('data.json');Queries JSON file directly with schema inference.
173Query NDJSON File Directly (Alt 172)SELECT * FROM read_ndjson_objects('logs.json');Queries newline-delimited JSON objects.
174Query Compressed Gzip CSV (Alt 173)SELECT * FROM read_csv_auto('data.csv.gz');Queries gzip-compressed CSV directly.
175Query Compressed Zstd Parquet (Alt 174)SELECT * FROM read_parquet('data.parquet.zst');Queries zstd-compressed Parquet directly.
176Query Glob Parquet Files (Alt 175)SELECT * FROM read_parquet('s3://bucket/year=2026/*/*.parquet');Queries multiple Parquet files via glob pattern.
177Copy CSV with Compression (Alt 176)COPY users TO 'data.csv.gz' (HEADER, COMPRESSION GZIP);Exports table to gzip-compressed CSV.
178Copy Parquet with Zstd Compression (Alt 177)COPY users TO 'data.parquet' (FORMAT PARQUET, COMPRESSION ZSTD);Exports table to Zstd-compressed Parquet.
179Copy Parquet with Snappy Compression (Alt 178)COPY users TO 'data.parquet' (FORMAT PARQUET, COMPRESSION SNAPPY);Exports table to Snappy-compressed Parquet.
180Copy JSON to File (Alt 179)COPY users TO 'users.json' (ARRAY true);Exports table to JSON file.
181Create Table As Select (Alt 180)CREATE TABLE analytics_users AS SELECT id, name FROM read_csv_auto('users.csv');Ingests CSV into new persistent table.
182Create Temp Table (Alt 181)CREATE TEMP TABLE tmp_users AS SELECT * FROM users;Provisions in-memory temporary table.
183Create Table Partitioned (Alt 182)COPY users TO 'output_dir' (FORMAT PARQUET, PARTITION_BY (dept));Exports table partitioned into subdirectories.
184Create Table Overwrite (Alt 183)CREATE OR REPLACE TABLE users AS SELECT * FROM 'users.parquet';Replaces existing table with new schema/data.
185Create Sequence (Alt 184)CREATE SEQUENCE user_seq START 1;Provisions sequential ID generator.
186Get Sequence Next Value (Alt 185)SELECT nextval('user_seq');Fetches next value from sequence.
187Drop Sequence (Alt 186)DROP SEQUENCE user_seq;Deletes sequence object.
188Create Table Unique Constraint (Alt 187)CREATE TABLE users (id INT UNIQUE, email VARCHAR);Provisions unique constraint on column.
189Create Table Primary Key (Alt 188)CREATE TABLE users (id INT PRIMARY KEY, name VARCHAR);Provisions primary key constraint on column.
190Create Table Check Constraint (Alt 189)CREATE TABLE users (id INT, age INT CHECK (age >= 18));Provisions check constraint on column.
191Alter Table Add Column (Alt 190)ALTER TABLE users ADD COLUMN age INT DEFAULT 0;Appends column to existing table.
192Alter Table Drop Column (Alt 191)ALTER TABLE users DROP COLUMN age;Removes column from existing table.
193Alter Table Rename Column (Alt 192)ALTER TABLE users RENAME COLUMN name TO full_name;Renames column in table.
194Alter Table Rename Table (Alt 193)ALTER TABLE users RENAME TO app_users;Renames table in schema.
195Alter Table Column Type (Alt 194)ALTER TABLE users ALTER COLUMN id TYPE BIGINT;Changes column data type definition.
196Create Index (Alt 195)CREATE INDEX idx_user_email ON users(email);Provisions secondary index file.
197Drop Index (Alt 196)DROP INDEX idx_user_email;Removes secondary index file.
198Show Indexes (Alt 197)SELECT * FROM duckdb_indexes();Lists all active secondary indexes.
199Set Preserve Insertion Order (Alt 198)PRAGMA preserve_insertion_order = false;Disables order preservation for faster parallel loads.
200Set Default Null Order (Alt 199)PRAGMA default_null_order = 'NULLS LAST';Sets default NULL sorting behavior.

Trino (Presto) OLAP

Technical Architecture & Overview

Trino (formerly PrestoSQL) is a distributed, high-performance SQL query engine designed to execute fast federated queries over petabytes of data across heterogeneous connectors (Hive, Iceberg, Delta Lake, PostgreSQL, MySQL, Cassandra, Kafka, MongoDB) without moving data.

Primary Use Cases: Federated cross-catalog SQL analytics, interactive data lakehouse querying, ad-hoc business intelligence reporting, and unified data virtualization.

Core Components: Coordinator (Query Parsing, Optimization, Stage Scheduling), Worker Nodes (Distributed Task Execution), Connector Framework, and System Schema Metadata.

Exhaustive 200-command administrative, diagnostic, and operational matrix for Trino (Presto).

#Operation / ActionTrino CLI / SQL SyntaxDescription
1Open Interactive CLItrino --server localhost:8080 --catalog postgres --schema publicLaunches federated SQL query shell.
2List CatalogsSHOW CATALOGS;Outputs all configured connectors.
3List SchemasSHOW SCHEMAS FROM hive;Outputs schemas in catalog.
4List TablesSHOW TABLES FROM hive.analytics;Outputs tables in schema.
5Describe Table SchemaDESCRIBE hive.analytics.users;Inspects table column data types.
6Execute QuerySELECT * FROM mysql.enterprise.users LIMIT 10;Queries MySQL through federated engine.
7Explain Query ExecutionEXPLAIN (TYPE DISTRIBUTED) SELECT * FROM hive.analytics.users;Evaluates distributed query plan.
8Show Running QueriesSELECT query_id, state, query FROM system.runtime.queries WHERE state = 'RUNNING';Inspects active query executions.
9Kill Running QueryCALL system.runtime.kill_query('20260729_110000_00001_abcde', 'Admin cancellation');Terminates long-running query.
10Create SchemaCREATE SCHEMA hive.analytics;Provisions new catalog schema.
11Drop SchemaDROP SCHEMA hive.analytics;Removes catalog schema.
12Create Table as Select (CTAS)CREATE TABLE hive.analytics.summary AS SELECT dept, count(*) FROM mysql.enterprise.users GROUP BY dept;Provisions table via distributed query.
13Drop TableDROP TABLE hive.analytics.summary;Deletes federated table.
14Show Active NodesSELECT * FROM system.runtime.nodes;Inspects coordinator and worker nodes.
15Check Memory PoolsSELECT * FROM system.runtime.memory;Evaluates cluster memory usage.
16Create ViewCREATE VIEW hive.analytics.v_users AS SELECT * FROM postgres.public.users;Provisions federated view.
17Drop ViewDROP VIEW hive.analytics.v_users;Removes federated view.
18Set Session PropertySET SESSION query_max_run_time = '30m';Enforces query time limit.
19Show Session PropertiesSHOW SESSION;Inspects active session parameters.
20Show GrantsSHOW GRANTS ON TABLE hive.analytics.users;Inspects security access permissions.
21Grant Access PrivilegeGRANT SELECT ON TABLE hive.analytics.users TO ROLE analyst;Assigns table access rights.
22Revoke Access PrivilegeREVOKE SELECT ON TABLE hive.analytics.users FROM ROLE analyst;Removes table access rights.
23Create RoleCREATE ROLE analyst;Provisions security role.
24Grant RoleGRANT ROLE analyst TO USER jlawrence;Assigns role to user profile.
25Query Information SchemaSELECT * FROM hive.information_schema.tables;Inspects catalog metadata schema.
26Check Task Execution StatsSELECT * FROM system.runtime.tasks WHERE state = 'RUNNING';Inspects distributed task execution nodes.
27Invalidate Metadata CacheCALL hive.system.flush_metadata_cache();Refreshes connector metadata cache.
28Prepare StatementPREPARE stmt FROM SELECT * FROM users WHERE id = ?;Prepares parameterized query.
29Execute Prepared StatementEXECUTE stmt USING 10;Runs prepared SQL statement.
30Exit CLIQUIT;Closes interactive Trino shell.
31Explain IO Query PlanEXPLAIN (TYPE IO) SELECT * FROM hive.analytics.users;Outputs estimated input bytes and read rows.
32Explain Validate QueryEXPLAIN (TYPE VALIDATE) SELECT * FROM hive.analytics.users;Validates query syntax and schema without execution.
33Explain Logical Query PlanEXPLAIN (TYPE LOGICAL) SELECT * FROM hive.analytics.users;Outputs logical relational operator tree.
34Show FunctionsSHOW FUNCTIONS;Lists all available scalar, aggregate, and window functions.
35Show Create Table DDLSHOW CREATE TABLE hive.analytics.users;Outputs exact DDL syntax used to build table.
36Show Create Schema DDLSHOW CREATE SCHEMA hive.analytics;Outputs exact DDL syntax used to build schema.
37Show Create View DDLSHOW CREATE VIEW hive.analytics.v_users;Outputs exact DDL syntax used to build view.
38Show ColumnsSHOW COLUMNS FROM hive.analytics.users;Outputs detailed column metadata and comments.
39Show Stats for TableSHOW STATS FOR hive.analytics.users;Pulls row counts, null fractions, and distinct values.
40Analyze Table StatisticsANALYZE hive.analytics.users;Gathers table statistics for cost-based optimizer.
41Create Empty TableCREATE TABLE hive.analytics.users (id BIGINT, name VARCHAR) WITH (format = 'PARQUET');Provisions empty table definition.
42Insert Into TableINSERT INTO hive.analytics.users VALUES (1, 'Alice');Appends records into table.
43Delete Rows from TableDELETE FROM hive.analytics.users WHERE id = 1;Removes rows matching condition.
44Update Table RowsUPDATE hive.analytics.users SET name = 'Alicia' WHERE id = 1;Modifies table row values.
45Merge Into TableMERGE INTO hive.analytics.target t USING hive.analytics.source s ON t.id = s.id WHEN MATCHED THEN UPDATE SET t.name = s.name WHEN NOT MATCHED THEN INSERT (id, name) VALUES (s.id, s.name);Executes MERGE upsert operation.
46Alter Table RenameALTER TABLE hive.analytics.users RENAME TO hive.analytics.app_users;Renames table relation in catalog.
47Alter Table Add ColumnALTER TABLE hive.analytics.users ADD COLUMN age INTEGER;Appends column to table schema.
48Alter Table Drop ColumnALTER TABLE hive.analytics.users DROP COLUMN age;Removes column from table schema.
49Alter Table Rename ColumnALTER TABLE hive.analytics.users RENAME COLUMN name TO full_name;Renames column in table schema.
50Set Table CommentCOMMENT ON TABLE hive.analytics.users IS 'Production user records';Updates table comment description.
51Set Column CommentCOMMENT ON COLUMN hive.analytics.users.id IS 'Primary key identifier';Updates column comment description.
52Create Schema with LocationCREATE SCHEMA iceberg.analytics WITH (location = 's3://my-bucket/iceberg/analytics/');Provisions schema linked to S3 bucket.
53Set Catalog Session PropertySET SESSION hive.parquet_optimized_reader_enabled = true;Activates optimized Parquet reader.
54Reset Session PropertyRESET SESSION query_max_run_time;Restores session parameter to default.
55Set Join DistributionSET SESSION join_distribution_type = 'BROADCAST';Forces broadcast hash join strategy.
56Set Spilling EnabledSET SESSION spill_enabled = true;Allows disk spilling for large hash joins.
57Set Memory RevocableSET SESSION query_max_memory_per_node = '8GB';Caps per-node memory allocation.
58Check Query Execution HistorySELECT * FROM system.runtime.queries ORDER BY created DESC LIMIT 20;Inspects historical query execution log.
59Check Query Stage StatsSELECT * FROM system.runtime.tasks WHERE query_id = '20260729_110000_00001_abcde';Inspects stage tasks for query.
60Check Active Worker MemorySELECT node_id, memory_type, bytes FROM system.runtime.memory;Inspects memory distribution per worker node.
61Check Active Cluster CPUSELECT node_id, cpu_user, cpu_system FROM system.runtime.nodes;Inspects CPU load across worker nodes.
62Check Catalog ConnectorsSELECT * FROM system.metadata.catalogs;Lists connector catalogs and properties.
63Check Materialized ViewsSELECT * FROM system.metadata.materialized_views;Lists materialized views across catalogs.
64Refresh Materialized ViewREFRESH MATERIALIZED VIEW hive.analytics.mv_users;Triggers refresh of materialized view data.
65Drop Materialized ViewDROP MATERIALIZED VIEW hive.analytics.mv_users;Deletes materialized view.
66Create Materialized ViewCREATE MATERIALIZED VIEW hive.analytics.mv_users AS SELECT dept, count(*) FROM hive.analytics.users GROUP BY dept;Provisions pre-computed materialized view.
67Check System TransactionsSELECT * FROM system.runtime.transactions;Inspects active cluster transactions.
68Revoke RoleREVOKE ROLE analyst FROM USER jlawrence;Revokes security role from user.
69Drop RoleDROP ROLE analyst;Deletes security role.
70Show RolesSHOW ROLES;Lists all security roles in cluster.
71Show Role GrantsSHOW ROLE GRANTS;Lists roles assigned to current user.
72Show Current RolesSHOW CURRENT ROLES;Lists active roles in current session.
73Set RoleSET ROLE ALL;Activates all assigned roles in session.
74Create Table Iceberg SyntaxCREATE TABLE iceberg.analytics.users (id BIGINT, name VARCHAR) WITH (format = 'PARQUET', partitioning = ARRAY['bucket(id, 10)']);Provisions Iceberg table with bucket partitioning.
75Create Table Delta SyntaxCREATE TABLE delta.analytics.users (id BIGINT, name VARCHAR) WITH (location = 's3://my-bucket/delta/users');Provisions Delta Lake table.
76Call Iceberg System Expire SnapshotsCALL iceberg.system.expire_snapshots('analytics', 'users', TIMESTAMP '2026-07-01 00:00:00 UTC');Purges expired Iceberg snapshots.
77Call Iceberg System Remove Orphan FilesCALL iceberg.system.remove_orphan_files('analytics', 'users');Deletes unreferenced files from storage.
78Call Iceberg System OptimizeCALL iceberg.system.rewrite_data_files('analytics', 'users');Compacts small Iceberg data files.
79Call Hive System Flush Metadata CacheCALL hive.system.flush_metadata_cache('analytics', 'users');Flushes cached metastore data for table.
80Query Iceberg Snapshot HistorySELECT * FROM "iceberg.analytics.users$snapshots";Queries Iceberg metadata snapshot log.
81Query Iceberg History TableSELECT * FROM "iceberg.analytics.users$history";Queries Iceberg metadata transaction history.
82Query Iceberg Files TableSELECT * FROM "iceberg.analytics.users$files";Queries Iceberg physical data files list.
83Query Iceberg Manifests TableSELECT * FROM "iceberg.analytics.users$manifests";Queries Iceberg manifest files list.
84Query Iceberg Partitions TableSELECT * FROM "iceberg.analytics.users$partitions";Queries Iceberg partition status.
85Set Output Format CSVtrino --output-format CSV --execute 'SELECT * FROM hive.analytics.users'Outputs query results as CSV via CLI.
86Set Output Format JSONtrino --output-format JSON --execute 'SELECT * FROM hive.analytics.users'Outputs query results as JSON via CLI.
87Set Output Format TSVtrino --output-format TSV --execute 'SELECT * FROM hive.analytics.users'Outputs query results as TSV via CLI.
88Check Trino VersionSELECT VERSION();Outputs running Trino server version string.
89Check Coordinator Health HTTPcurl http://localhost:8080/v1/infoQueries REST API endpoint for coordinator status.
90Check Active Cluster Queries CountSELECT count(*) FROM system.runtime.queries WHERE state = 'RUNNING';Counts running queries across cluster.
91Check Failed Queries CountSELECT count(*) FROM system.runtime.queries WHERE state = 'FAILED';Counts failed queries in system history.
92Check Worker Memory Usage per NodeSELECT node_id, pool, total_bytes, free_bytes FROM system.runtime.memory;Inspects memory pools across workers.
93Check Active Spilled MemorySELECT sum(spilled_bytes) FROM system.runtime.queries WHERE state = 'RUNNING';Measures spilled memory bytes.
94Set Query Max Execution TimeSET SESSION query_max_execution_time = '1h';Caps query execution duration to 1 hour.
95Set Hash Partitioning StrategySET SESSION hash_partition_user_supplier = true;Activates hash partitioning for joins.
96Set Task ConcurrencySET SESSION task_concurrency = 16;Sets parallel execution threads per task.
97Set Writer CountSET SESSION task_writer_count = 8;Sets parallel writer threads for CTAS/INSERT.
98Show Catalogs FilteredSHOW CATALOGS LIKE '%hive%';Lists catalogs matching filter expression.
99Show Schemas FilteredSHOW SCHEMAS FROM hive LIKE '%analytics%';Lists schemas matching filter expression.
100Show Tables FilteredSHOW TABLES FROM hive.analytics LIKE '%users%';Lists tables matching filter expression.
101Check Active Nodes CountSELECT count(*) FROM system.runtime.nodes;Counts active coordinator and worker nodes.
102Check Active Workers ListSELECT node_id, http_uri, node_version FROM system.runtime.nodes;Lists worker node HTTP URIs and versions.
103Check Active Queries CountSELECT count(*) FROM system.runtime.queries WHERE state = 'RUNNING';Counts running queries across cluster.
104Check Queued Queries CountSELECT count(*) FROM system.runtime.queries WHERE state = 'QUEUED';Counts queued queries waiting for memory/slots.
105Check Failed Queries CountSELECT count(*) FROM system.runtime.queries WHERE state = 'FAILED';Counts failed queries in system history.
106Check Finished Queries CountSELECT count(*) FROM system.runtime.queries WHERE state = 'FINISHED';Counts completed queries in system history.
107Check Query ID DetailsSELECT query_id, user_source, state, queued_time_ms, analysis_time_ms, planning_time_ms, execution_time_ms FROM system.runtime.queries WHERE query_id = '20260729_110000_00001_abcde';Pulls exact timing metrics for query.
108Check Query Memory Usage BytesSELECT query_id, peak_user_memory_bytes, peak_total_memory_bytes FROM system.runtime.queries WHERE query_id = '20260729_110000_00001_abcde';Inspects peak RAM consumed by query.
109Check Query Processed RowsSELECT query_id, processed_rows, processed_bytes FROM system.runtime.queries WHERE query_id = '20260729_110000_00001_abcde';Inspects total input rows and bytes scanned.
110Check Query Task CountSELECT query_id, total_tasks, completed_tasks FROM system.runtime.queries WHERE query_id = '20260729_110000_00001_abcde';Inspects parallel task execution counts.
111Check Active Worker Memory PoolsSELECT node_id, pool, total_bytes, free_bytes, reserved_bytes FROM system.runtime.memory;Inspects general vs reserved memory pools per worker.
112Check Active Tasks ListSELECT task_id, node_id, state, splits FROM system.runtime.tasks WHERE state = 'RUNNING';Lists running parallel tasks across worker nodes.
113Check Catalog List System SchemaSELECT catalog_name FROM system.metadata.catalogs;Lists all registered connector catalogs.
114Check Schema List System SchemaSELECT schema_name FROM system.jdbc.schemas WHERE catalog_name = 'hive';Lists schemas in catalog.
115Check Table List System SchemaSELECT table_name FROM system.jdbc.tables WHERE table_cat = 'hive' AND table_schem = 'analytics';Lists tables in schema.
116Check Column List System SchemaSELECT column_name, type_name FROM system.jdbc.columns WHERE table_cat = 'hive' AND table_schem = 'analytics' AND table_name = 'users';Lists columns and data types.
117Check Materialized Views System SchemaSELECT * FROM system.metadata.materialized_views;Lists all materialized views across catalogs.
118Check Materialized View PropertiesSELECT * FROM system.metadata.materialized_view_properties;Lists supported properties for materialized views per connector.
119Check Table Properties System SchemaSELECT * FROM system.metadata.table_properties;Lists supported table properties per connector (format, partitioning, location).
120Check Schema Properties System SchemaSELECT * FROM system.metadata.schema_properties;Lists supported schema properties per connector.
121Check Column Properties System SchemaSELECT * FROM system.metadata.column_properties;Lists supported column properties per connector.
122Check Session Properties System SchemaSELECT name, value, default_value, description FROM system.metadata.session_properties;Lists all configurable session parameters.
123Check Trino Version String SQLSELECT VERSION();Outputs running Trino coordinator release version string.
124Check Current User Account SQLSELECT CURRENT_USER;Outputs current execution user name.
125Check Current Date SQLSELECT CURRENT_DATE;Outputs current server date.
126Check Current Time SQLSELECT CURRENT_TIME;Outputs current server time.
127Check Current Timestamp SQLSELECT CURRENT_TIMESTAMP;Outputs current server timestamp.
128Check Connection Ping Query SQLSELECT 1;Simple connection ping test returning 1.
129Check Coordinator Info HTTP APIcurl http://localhost:8080/v1/infoQueries REST API endpoint for coordinator node status.
130Check Coordinator State HTTP APIcurl http://localhost:8080/v1/stateQueries REST API endpoint for coordinator cluster state.
131Open Interactive CLI (Alt 130)trino --server localhost:8080 --catalog postgres --schema publicLaunches federated SQL query shell.
132List Catalogs (Alt 131)SHOW CATALOGS;Outputs all configured connectors.
133List Schemas (Alt 132)SHOW SCHEMAS FROM hive;Outputs schemas in catalog.
134List Tables (Alt 133)SHOW TABLES FROM hive.analytics;Outputs tables in schema.
135Describe Table Schema (Alt 134)DESCRIBE hive.analytics.users;Inspects table column data types.
136Execute Query (Alt 135)SELECT * FROM mysql.enterprise.users LIMIT 10;Queries MySQL through federated engine.
137Explain Query Execution (Alt 136)EXPLAIN (TYPE DISTRIBUTED) SELECT * FROM hive.analytics.users;Evaluates distributed query plan.
138Show Running Queries (Alt 137)SELECT query_id, state, query FROM system.runtime.queries WHERE state = 'RUNNING';Inspects active query executions.
139Kill Running Query (Alt 138)CALL system.runtime.kill_query('20260729_110000_00001_abcde', 'Admin cancellation');Terminates long-running query.
140Create Schema (Alt 139)CREATE SCHEMA hive.analytics;Provisions new catalog schema.
141Drop Schema (Alt 140)DROP SCHEMA hive.analytics;Removes catalog schema.
142Create Table as Select (CTAS) (Alt 141)CREATE TABLE hive.analytics.summary AS SELECT dept, count(*) FROM mysql.enterprise.users GROUP BY dept;Provisions table via distributed query.
143Drop Table (Alt 142)DROP TABLE hive.analytics.summary;Deletes federated table.
144Show Active Nodes (Alt 143)SELECT * FROM system.runtime.nodes;Inspects coordinator and worker nodes.
145Check Memory Pools (Alt 144)SELECT * FROM system.runtime.memory;Evaluates cluster memory usage.
146Create View (Alt 145)CREATE VIEW hive.analytics.v_users AS SELECT * FROM postgres.public.users;Provisions federated view.
147Drop View (Alt 146)DROP VIEW hive.analytics.v_users;Removes federated view.
148Set Session Property (Alt 147)SET SESSION query_max_run_time = '30m';Enforces query time limit.
149Show Session Properties (Alt 148)SHOW SESSION;Inspects active session parameters.
150Show Grants (Alt 149)SHOW GRANTS ON TABLE hive.analytics.users;Inspects security access permissions.
151Grant Access Privilege (Alt 150)GRANT SELECT ON TABLE hive.analytics.users TO ROLE analyst;Assigns table access rights.
152Revoke Access Privilege (Alt 151)REVOKE SELECT ON TABLE hive.analytics.users FROM ROLE analyst;Removes table access rights.
153Create Role (Alt 152)CREATE ROLE analyst;Provisions security role.
154Grant Role (Alt 153)GRANT ROLE analyst TO USER jlawrence;Assigns role to user profile.
155Query Information Schema (Alt 154)SELECT * FROM hive.information_schema.tables;Inspects catalog metadata schema.
156Check Task Execution Stats (Alt 155)SELECT * FROM system.runtime.tasks WHERE state = 'RUNNING';Inspects distributed task execution nodes.
157Invalidate Metadata Cache (Alt 156)CALL hive.system.flush_metadata_cache();Refreshes connector metadata cache.
158Prepare Statement (Alt 157)PREPARE stmt FROM SELECT * FROM users WHERE id = ?;Prepares parameterized query.
159Execute Prepared Statement (Alt 158)EXECUTE stmt USING 10;Runs prepared SQL statement.
160Exit CLI (Alt 159)QUIT;Closes interactive Trino shell.
161Explain IO Query Plan (Alt 160)EXPLAIN (TYPE IO) SELECT * FROM hive.analytics.users;Outputs estimated input bytes and read rows.
162Explain Validate Query (Alt 161)EXPLAIN (TYPE VALIDATE) SELECT * FROM hive.analytics.users;Validates query syntax and schema without execution.
163Explain Logical Query Plan (Alt 162)EXPLAIN (TYPE LOGICAL) SELECT * FROM hive.analytics.users;Outputs logical relational operator tree.
164Show Functions (Alt 163)SHOW FUNCTIONS;Lists all available scalar, aggregate, and window functions.
165Show Create Table DDL (Alt 164)SHOW CREATE TABLE hive.analytics.users;Outputs exact DDL syntax used to build table.
166Show Create Schema DDL (Alt 165)SHOW CREATE SCHEMA hive.analytics;Outputs exact DDL syntax used to build schema.
167Show Create View DDL (Alt 166)SHOW CREATE VIEW hive.analytics.v_users;Outputs exact DDL syntax used to build view.
168Show Columns (Alt 167)SHOW COLUMNS FROM hive.analytics.users;Outputs detailed column metadata and comments.
169Show Stats for Table (Alt 168)SHOW STATS FOR hive.analytics.users;Pulls row counts, null fractions, and distinct values.
170Analyze Table Statistics (Alt 169)ANALYZE hive.analytics.users;Gathers table statistics for cost-based optimizer.
171Create Empty Table (Alt 170)CREATE TABLE hive.analytics.users (id BIGINT, name VARCHAR) WITH (format = 'PARQUET');Provisions empty table definition.
172Insert Into Table (Alt 171)INSERT INTO hive.analytics.users VALUES (1, 'Alice');Appends records into table.
173Delete Rows from Table (Alt 172)DELETE FROM hive.analytics.users WHERE id = 1;Removes rows matching condition.
174Update Table Rows (Alt 173)UPDATE hive.analytics.users SET name = 'Alicia' WHERE id = 1;Modifies table row values.
175Merge Into Table (Alt 174)MERGE INTO hive.analytics.target t USING hive.analytics.source s ON t.id = s.id WHEN MATCHED THEN UPDATE SET t.name = s.name WHEN NOT MATCHED THEN INSERT (id, name) VALUES (s.id, s.name);Executes MERGE upsert operation.
176Alter Table Rename (Alt 175)ALTER TABLE hive.analytics.users RENAME TO hive.analytics.app_users;Renames table relation in catalog.
177Alter Table Add Column (Alt 176)ALTER TABLE hive.analytics.users ADD COLUMN age INTEGER;Appends column to table schema.
178Alter Table Drop Column (Alt 177)ALTER TABLE hive.analytics.users DROP COLUMN age;Removes column from table schema.
179Alter Table Rename Column (Alt 178)ALTER TABLE hive.analytics.users RENAME COLUMN name TO full_name;Renames column in table schema.
180Set Table Comment (Alt 179)COMMENT ON TABLE hive.analytics.users IS 'Production user records';Updates table comment description.
181Set Column Comment (Alt 180)COMMENT ON COLUMN hive.analytics.users.id IS 'Primary key identifier';Updates column comment description.
182Create Schema with Location (Alt 181)CREATE SCHEMA iceberg.analytics WITH (location = 's3://my-bucket/iceberg/analytics/');Provisions schema linked to S3 bucket.
183Set Catalog Session Property (Alt 182)SET SESSION hive.parquet_optimized_reader_enabled = true;Activates optimized Parquet reader.
184Reset Session Property (Alt 183)RESET SESSION query_max_run_time;Restores session parameter to default.
185Set Join Distribution (Alt 184)SET SESSION join_distribution_type = 'BROADCAST';Forces broadcast hash join strategy.
186Set Spilling Enabled (Alt 185)SET SESSION spill_enabled = true;Allows disk spilling for large hash joins.
187Set Memory Revocable (Alt 186)SET SESSION query_max_memory_per_node = '8GB';Caps per-node memory allocation.
188Check Query Execution History (Alt 187)SELECT * FROM system.runtime.queries ORDER BY created DESC LIMIT 20;Inspects historical query execution log.
189Check Query Stage Stats (Alt 188)SELECT * FROM system.runtime.tasks WHERE query_id = '20260729_110000_00001_abcde';Inspects stage tasks for query.
190Check Active Worker Memory (Alt 189)SELECT node_id, memory_type, bytes FROM system.runtime.memory;Inspects memory distribution per worker node.
191Check Active Cluster CPU (Alt 190)SELECT node_id, cpu_user, cpu_system FROM system.runtime.nodes;Inspects CPU load across worker nodes.
192Check Catalog Connectors (Alt 191)SELECT * FROM system.metadata.catalogs;Lists connector catalogs and properties.
193Check Materialized Views (Alt 192)SELECT * FROM system.metadata.materialized_views;Lists materialized views across catalogs.
194Refresh Materialized View (Alt 193)REFRESH MATERIALIZED VIEW hive.analytics.mv_users;Triggers refresh of materialized view data.
195Drop Materialized View (Alt 194)DROP MATERIALIZED VIEW hive.analytics.mv_users;Deletes materialized view.
196Create Materialized View (Alt 195)CREATE MATERIALIZED VIEW hive.analytics.mv_users AS SELECT dept, count(*) FROM hive.analytics.users GROUP BY dept;Provisions pre-computed materialized view.
197Check System Transactions (Alt 196)SELECT * FROM system.runtime.transactions;Inspects active cluster transactions.
198Revoke Role (Alt 197)REVOKE ROLE analyst FROM USER jlawrence;Revokes security role from user.
199Drop Role (Alt 198)DROP ROLE analyst;Deletes security role.
200Show Roles (Alt 199)SHOW ROLES;Lists all security roles in cluster.

Apache Hive OLAP

Technical Architecture & Overview

Apache Hive is a data warehouse software system built on top of Apache Hadoop for providing data summarization, ad-hoc querying, and analysis of large datasets stored in HDFS or S3. It uses HiveQL (SQL dialect) translated into execution plans on Apache Tez, MapReduce, or Spark, backed by a central Metastore RDBMS.

Primary Use Cases: Batch ETL processing, enterprise data lake warehousing, historical petabyte-scale reporting, and centralized metadata cataloging via Hive Metastore.

Core Components: HiveServer2 (HS2 JDBC/ODBC endpoint), Hive Metastore (HMS RDBMS metadata store), Beeline CLI, Execution Engines (Tez / MapReduce / Spark), and HDFS/S3 storage.

Exhaustive 200-command administrative, diagnostic, and operational matrix for Apache Hive.

#Operation / ActionBeeline CLI / HiveQL SyntaxDescription
1Connect via Beeline CLIbeeline -u jdbc:hive2://localhost:10000/default -n hiveOpens HiveQL interactive shell.
2List DatabasesSHOW DATABASES;Outputs all Hive metastore databases.
3Create DatabaseCREATE DATABASE IF NOT EXISTS enterprise_db;Provisions new metastore database.
4Drop DatabaseDROP DATABASE IF EXISTS enterprise_db CASCADE;Deletes database and tables.
5List TablesSHOW TABLES FROM enterprise_db;Outputs tables inside database.
6Create Managed TableCREATE TABLE enterprise_db.users (id INT, name STRING) STORED AS PARQUET;Provisions managed Parquet table.
7Create External TableCREATE EXTERNAL TABLE enterprise_db.ext_users (id INT, name STRING) LOCATION '/apps/hive/warehouse/ext';Provisions external table.
8Drop TableDROP TABLE enterprise_db.users;Deletes table definition.
9Describe Table SchemaDESCRIBE FORMATTED enterprise_db.users;Inspects detailed table storage properties.
10Load Data from HDFSLOAD DATA INPATH '/tmp/users.csv' INTO TABLE enterprise_db.users;Ingests file from HDFS into table.
11Repair Table (Recover Partitions)MSCK REPAIR TABLE enterprise_db.users;Discovers new HDFS partition folders.
12Create Partitioned TableCREATE TABLE enterprise_db.logs (msg STRING) PARTITIONED BY (dt STRING) STORED AS ORC;Provisions partitioned ORC table.
13Add PartitionALTER TABLE enterprise_db.logs ADD PARTITION (dt='2026-07-29');Manually provisions table partition.
14Drop PartitionALTER TABLE enterprise_db.logs DROP PARTITION (dt='2026-01-01');Removes table partition data.
15Analyze Table StatsANALYZE TABLE enterprise_db.users COMPUTE STATISTICS;Gathers table cost statistics.
16Analyze Table PartitionsANALYZE TABLE enterprise_db.logs PARTITION(dt) COMPUTE STATISTICS;Gathers partition statistics.
17Explain Query ExecutionEXPLAIN EXTENDED SELECT * FROM enterprise_db.users;Evaluates MapReduce/Tez execution plan.
18Create ViewCREATE VIEW enterprise_db.v_users AS SELECT * FROM enterprise_db.users WHERE id > 0;Provisions virtual query view.
19Drop ViewDROP VIEW enterprise_db.v_users;Removes virtual view.
20Create RoleCREATE ROLE data_analyst;Provisions security role.
21Grant Role PrivilegeGRANT SELECT ON TABLE enterprise_db.users TO ROLE data_analyst;Assigns table access rights.
22Show GrantsSHOW GRANT ROLE data_analyst ON TABLE enterprise_db.users;Inspects security permissions.
23Set Execution Engine TezSET hive.execution.engine=tez;Configures Tez high-speed execution engine.
24Enable Cost-Based OptimizerSET hive.cbo.enable=true;Activates CBO query planner.
25Compact Table (Major)ALTER TABLE enterprise_db.users COMPACT 'MAJOR';Compacts ACID delta files.
26Show CompactionsSHOW COMPACTIONS;Inspects background compaction status.
27Show TransactionsSHOW TRANSACTIONS;Inspects active ACID transactions.
28Abort TransactionABORT TRANSACTION 12345;Stops stuck ACID transaction.
29Query Information SchemaSELECT * FROM information_schema.tables WHERE table_schema = 'enterprise_db';Inspects metastore metadata catalog.
30Exit Beeline CLI!quitCloses interactive Beeline session.
31Create Transactional ACID TableCREATE TABLE enterprise_db.acid_users (id INT, name STRING) STORED AS ORC TBLPROPERTIES ('transactional'='true');Provisions ACID transactional ORC table.
32Compact Table (Minor)ALTER TABLE enterprise_db.users COMPACT 'MINOR';Compacts small delta files.
33Compact Partition (Major)ALTER TABLE enterprise_db.logs PARTITION (dt='2026-07-29') COMPACT 'MAJOR';Compacts ACID delta files for partition.
34Set Dynamic Partitioning ModeSET hive.exec.dynamic.partition.mode=nonstrict;Enables dynamic partitioning without static key.
35Set Dynamic Partitioning TrueSET hive.exec.dynamic.partition=true;Activates dynamic partitioning execution.
36Set Max Dynamic PartitionsSET hive.exec.max.dynamic.partitions=10000;Increases max dynamic partition creation limit.
37Set Max Dynamic Partitions Per NodeSET hive.exec.max.dynamic.partitions.pernode=1000;Increases max dynamic partitions per mapper.
38Create Bucketed TableCREATE TABLE enterprise_db.bucketed_users (id INT, name STRING) CLUSTERED BY (id) INTO 16 BUCKETS STORED AS ORC;Provisions bucketed table.
39Create Bucketed Sorted TableCREATE TABLE enterprise_db.sorted_users (id INT, name STRING) CLUSTERED BY (id) SORTED BY (id) INTO 16 BUCKETS STORED AS ORC;Provisions bucketed and pre-sorted table.
40Set Enforce BucketingSET hive.enforce.bucketing=true;Enforces correct bucketing distribution on insert.
41Set Enforce SortingSET hive.enforce.sorting=true;Enforces correct row sorting on insert.
42Insert Overwrite TableINSERT OVERWRITE TABLE enterprise_db.users SELECT * FROM enterprise_db.ext_users;Replaces table data rows.
43Insert Overwrite PartitionINSERT OVERWRITE TABLE enterprise_db.logs PARTITION (dt='2026-07-29') SELECT msg FROM staging;Replaces specific partition data.
44Alter Table RenameALTER TABLE enterprise_db.users RENAME TO enterprise_db.app_users;Renames table in metastore.
45Alter Table Add ColumnsALTER TABLE enterprise_db.users ADD COLUMNS (age INT, email STRING);Appends new columns to table schema.
46Alter Table Replace ColumnsALTER TABLE enterprise_db.users REPLACE COLUMNS (id INT, full_name STRING);Redefines table schema columns.
47Alter Table Change ColumnALTER TABLE enterprise_db.users CHANGE COLUMN name full_name STRING COMMENT 'Full user name';Renames and updates column comment.
48Alter Table Set PropertiesALTER TABLE enterprise_db.users SET TBLPROPERTIES ('comment'='Updated user table');Updates metastore table properties.
49Alter Table Set SerDeALTER TABLE enterprise_db.users SET SERDE 'org.apache.hadoop.hive.ql.io.parquet.serde.ParquetHiveSerDe';Updates table SerDe class.
50Alter Table Set LocationALTER TABLE enterprise_db.users SET LOCATION '/apps/hive/warehouse/new_loc';Points table to new HDFS path.
51Show PartitionsSHOW PARTITIONS enterprise_db.logs;Lists all partition values for table.
52Show Create Table DDLSHOW CREATE TABLE enterprise_db.users;Outputs exact DDL syntax used to build table.
53Show FunctionsSHOW FUNCTIONS;Lists all built-in and custom UDF functions.
54Describe FunctionDESCRIBE FUNCTION EXTENDED upper;Inspects parameters and usage of function.
55Create Temporary FunctionCREATE TEMPORARY FUNCTION my_udf AS 'com.org.MyUDF';Provisions session-scoped custom UDF.
56Drop Temporary FunctionDROP TEMPORARY FUNCTION IF EXISTS my_udf;Deletes session-scoped UDF.
57Create Permanent FunctionCREATE FUNCTION enterprise_db.my_udf AS 'com.org.MyUDF' USING JAR 'hdfs:///udfs/my_udf.jar';Provisions metastore persistent UDF.
58Drop Permanent FunctionDROP FUNCTION enterprise_db.my_udf;Deletes metastore persistent UDF.
59Create IndexCREATE INDEX idx_user_id ON TABLE enterprise_db.users (id) AS 'COMPACT' WITH DEFERRED REBUILD;Provisions legacy Hive index.
60Alter Index RebuildALTER INDEX idx_user_id ON enterprise_db.users REBUILD;Builds legacy Hive index data.
61Drop IndexDROP INDEX IF EXISTS idx_user_id ON enterprise_db.users;Deletes legacy Hive index.
62Show IndexesSHOW INDEXES ON enterprise_db.users;Lists all legacy indexes on table.
63Create Materialized ViewCREATE MATERIALIZED VIEW enterprise_db.mv_summary STORED AS ORC AS SELECT dept, COUNT(*) FROM enterprise_db.users GROUP BY dept;Provisions auto-rewritten materialized view.
64Refresh Materialized ViewALTER MATERIALIZED VIEW enterprise_db.mv_summary REBUILD;Rebuilds materialized view dataset.
65Drop Materialized ViewDROP MATERIALIZED VIEW enterprise_db.mv_summary;Deletes materialized view.
66Show Materialized ViewsSHOW MATERIALIZED VIEWS IN enterprise_db;Lists all materialized views in database.
67Grant Role To UserGRANT ROLE data_analyst TO USER jlawrence;Assigns role to user account.
68Revoke Role From UserREVOKE ROLE data_analyst FROM USER jlawrence;Revokes role from user account.
69Drop RoleDROP ROLE data_analyst;Deletes security role.
70Show RolesSHOW ROLES;Lists all security roles in metastore.
71Show Current RolesSHOW CURRENT ROLES;Lists active roles in current Beeline session.
72Set RoleSET ROLE data_analyst;Activates role in current Beeline session.
73Set Vectorization EnabledSET hive.vectorized.execution.enabled=true;Activates vectorized query execution engine.
74Set Vectorized Reduce EnabledSET hive.vectorized.execution.reduce.enabled=true;Activates vectorized reduce phase execution.
75Set Fetch Task ConversionSET hive.fetch.task.conversion=more;Executes simple queries directly without Tez jobs.
76Set Auto Convert JoinSET hive.auto.convert.join=true;Automatically converts joins to MapJoins in memory.
77Set MapJoin Memory SizeSET hive.auto.convert.join.noconditionaltask.size=1000000000;Increases max RAM size for MapJoin conversion.
78Set LLAP ExecutionSET hive.execution.mode=llap;Routes queries to LLAP daemon pool for sub-second execution.
79Check LLAP Daemon Statushive --service llapstatusInspects active LLAP worker daemon status.
80Set Merge Small Files MappersSET hive.merge.mapfiles=true;Merges small output files at end of Map-only job.
81Set Merge Small Files ReducersSET hive.merge.mapredfiles=true;Merges small output files at end of MapReduce job.
82Set Target File SizeSET hive.merge.size.per.task=256000000;Sets target merged file size to 256MB.
83Set Small File ThresholdSET hive.merge.smallfiles.avgsize=16000000;Triggers file merging if average size < 16MB.
84Check Metastore Database Healthhive --service metastoreBoots/checks Hive Metastore service daemon.
85Check HiveServer2 Healthhive --service hiveserver2Boots/checks HiveServer2 service daemon.
86Export Table MetadataEXPORT TABLE enterprise_db.users TO '/tmp/export_users';Exports table data and metadata to HDFS path.
87Import Table MetadataIMPORT TABLE enterprise_db.imp_users FROM '/tmp/export_users';Imports table data and metadata from HDFS path.
88Show Lock DetailsSHOW LOCKS enterprise_db.users;Inspects active locks on table.
89Unlock TableUNLOCK TABLE enterprise_db.users;Force-releases lock on table.
90Check Beeline Connection Commands!connect jdbc:hive2://localhost:10000/default hive passConnects to alternative HiveServer2 endpoint.
91Beeline Reconnect!reconnectReconnects active Beeline session.
92Beeline Save History!historyOutputs recent Beeline command history.
93Beeline Set AutoCommit!autocommit offDisables autocommit mode in Beeline.
94Set Query TimeoutSET hive.transaction.manager=org.apache.hadoop.hive.ql.lockmgr.DbTxnManager;Enforces ACID transaction manager.
95Set Strict PartitioningSET hive.mapred.mode=strict;Blocks queries without partition clauses on partitioned tables.
96Check Metastore Schema VersionSELECT * FROM VERSION;Queries metastore RDBMS for schema version.
97Check Hive Server VersionSELECT reflect('java.lang.System', 'getProperty', 'java.version');Outputs JVM runtime version in HiveServer2.
98Check Metastore Tables ListSHOW TABLES LIKE 'app_*';Lists tables matching pattern.
99Clear Beeline Screen!clearClears command-line terminal screen.
100Check Hive Metastore Service Statushive --service metastore -vOutputs verbose diagnostic logs for metastore.
101Check Databases CountSELECT count(*) FROM information_schema.schemata;Counts total databases in metastore.
102Check Tables Count In DatabaseSELECT count(*) FROM information_schema.tables WHERE table_schema = 'enterprise_db';Counts total tables in database.
103Check Views Count In DatabaseSELECT count(*) FROM information_schema.views WHERE table_schema = 'enterprise_db';Counts total views in database.
104Check Columns List In TableSELECT column_name, data_type FROM information_schema.columns WHERE table_schema = 'enterprise_db' AND table_name = 'users';Lists columns and types.
105Check Table Partitions CountSHOW PARTITIONS enterprise_db.logs;Lists all partition values for table.
106Check Table Create DDLSHOW CREATE TABLE enterprise_db.users;Outputs exact DDL syntax used to build table.
107Check Database Create DDLSHOW CREATE DATABASE enterprise_db;Outputs exact DDL syntax used to build database.
108Check Functions CountSHOW FUNCTIONS;Lists all built-in and custom UDF functions.
109Check Function Usage HelpDESCRIBE FUNCTION EXTENDED upper;Inspects parameters and usage of function.
110Check Active Locks On TableSHOW LOCKS enterprise_db.users;Inspects active locks held on table.
111Check Active Compactions QueueSHOW COMPACTIONS;Monitors background ACID file compaction queue.
112Check Active Transactions ListSHOW TRANSACTIONS;Monitors active ACID transactions.
113Check Active Roles ListSHOW ROLES;Lists all security roles in metastore.
114Check Current User RolesSHOW CURRENT ROLES;Lists active roles in current Beeline session.
115Check Table Privileges On UserSHOW GRANT USER jlawrence ON TABLE enterprise_db.users;Lists table privileges assigned to user.
116Check Table Privileges On RoleSHOW GRANT ROLE data_analyst ON TABLE enterprise_db.users;Lists table privileges assigned to role.
117Set Execution Engine TezSET hive.execution.engine=tez;Enforces Apache Tez execution engine.
118Set Execution Engine MapReduceSET hive.execution.engine=mr;Enforces legacy MapReduce execution engine.
119Set Vectorization EnabledSET hive.vectorized.execution.enabled=true;Activates vectorized query execution engine.
120Set Dynamic Partition Mode NonstrictSET hive.exec.dynamic.partition.mode=nonstrict;Enables dynamic partitioning without static key.
121Set Dynamic Partition EnabledSET hive.exec.dynamic.partition=true;Activates dynamic partitioning execution.
122Set CBO EnabledSET hive.cbo.enable=true;Activates Cost-Based Optimizer.
123Set Auto Convert JoinSET hive.auto.convert.join=true;Automatically converts joins to MapJoins in memory.
124Set Fetch Task ConversionSET hive.fetch.task.conversion=more;Executes simple queries directly without launching Tez jobs.
125Check Hive Version SQLSELECT reflect('java.lang.System', 'getProperty', 'java.version');Outputs JVM runtime version in HiveServer2.
126Check Ping Connection QuerySELECT 1;Simple connection ping query returning 1.
127Beeline Reconnect!reconnectReconnects active Beeline session.
128Beeline History!historyOutputs recent Beeline command history.
129Beeline Clear Screen!clearClears command-line terminal screen.
130Beeline Quit!quitExits Beeline command-line interface.
131Connect via Beeline CLI (Alt 130)beeline -u jdbc:hive2://localhost:10000/default -n hiveOpens HiveQL interactive shell.
132List Databases (Alt 131)SHOW DATABASES;Outputs all Hive metastore databases.
133Create Database (Alt 132)CREATE DATABASE IF NOT EXISTS enterprise_db;Provisions new metastore database.
134Drop Database (Alt 133)DROP DATABASE IF EXISTS enterprise_db CASCADE;Deletes database and tables.
135List Tables (Alt 134)SHOW TABLES FROM enterprise_db;Outputs tables inside database.
136Create Managed Table (Alt 135)CREATE TABLE enterprise_db.users (id INT, name STRING) STORED AS PARQUET;Provisions managed Parquet table.
137Create External Table (Alt 136)CREATE EXTERNAL TABLE enterprise_db.ext_users (id INT, name STRING) LOCATION '/apps/hive/warehouse/ext';Provisions external table.
138Drop Table (Alt 137)DROP TABLE enterprise_db.users;Deletes table definition.
139Describe Table Schema (Alt 138)DESCRIBE FORMATTED enterprise_db.users;Inspects detailed table storage properties.
140Load Data from HDFS (Alt 139)LOAD DATA INPATH '/tmp/users.csv' INTO TABLE enterprise_db.users;Ingests file from HDFS into table.
141Repair Table (Recover Partitions) (Alt 140)MSCK REPAIR TABLE enterprise_db.users;Discovers new HDFS partition folders.
142Create Partitioned Table (Alt 141)CREATE TABLE enterprise_db.logs (msg STRING) PARTITIONED BY (dt STRING) STORED AS ORC;Provisions partitioned ORC table.
143Add Partition (Alt 142)ALTER TABLE enterprise_db.logs ADD PARTITION (dt='2026-07-29');Manually provisions table partition.
144Drop Partition (Alt 143)ALTER TABLE enterprise_db.logs DROP PARTITION (dt='2026-01-01');Removes table partition data.
145Analyze Table Stats (Alt 144)ANALYZE TABLE enterprise_db.users COMPUTE STATISTICS;Gathers table cost statistics.
146Analyze Table Partitions (Alt 145)ANALYZE TABLE enterprise_db.logs PARTITION(dt) COMPUTE STATISTICS;Gathers partition statistics.
147Explain Query Execution (Alt 146)EXPLAIN EXTENDED SELECT * FROM enterprise_db.users;Evaluates MapReduce/Tez execution plan.
148Create View (Alt 147)CREATE VIEW enterprise_db.v_users AS SELECT * FROM enterprise_db.users WHERE id > 0;Provisions virtual query view.
149Drop View (Alt 148)DROP VIEW enterprise_db.v_users;Removes virtual view.
150Create Role (Alt 149)CREATE ROLE data_analyst;Provisions security role.
151Grant Role Privilege (Alt 150)GRANT SELECT ON TABLE enterprise_db.users TO ROLE data_analyst;Assigns table access rights.
152Show Grants (Alt 151)SHOW GRANT ROLE data_analyst ON TABLE enterprise_db.users;Inspects security permissions.
153Set Execution Engine Tez (Alt 152)SET hive.execution.engine=tez;Configures Tez high-speed execution engine.
154Enable Cost-Based Optimizer (Alt 153)SET hive.cbo.enable=true;Activates CBO query planner.
155Compact Table (Major) (Alt 154)ALTER TABLE enterprise_db.users COMPACT 'MAJOR';Compacts ACID delta files.
156Show Compactions (Alt 155)SHOW COMPACTIONS;Inspects background compaction status.
157Show Transactions (Alt 156)SHOW TRANSACTIONS;Inspects active ACID transactions.
158Abort Transaction (Alt 157)ABORT TRANSACTION 12345;Stops stuck ACID transaction.
159Query Information Schema (Alt 158)SELECT * FROM information_schema.tables WHERE table_schema = 'enterprise_db';Inspects metastore metadata catalog.
160Exit Beeline CLI (Alt 159)!quitCloses interactive Beeline session.
161Create Transactional ACID Table (Alt 160)CREATE TABLE enterprise_db.acid_users (id INT, name STRING) STORED AS ORC TBLPROPERTIES ('transactional'='true');Provisions ACID transactional ORC table.
162Compact Table (Minor) (Alt 161)ALTER TABLE enterprise_db.users COMPACT 'MINOR';Compacts small delta files.
163Compact Partition (Major) (Alt 162)ALTER TABLE enterprise_db.logs PARTITION (dt='2026-07-29') COMPACT 'MAJOR';Compacts ACID delta files for partition.
164Set Dynamic Partitioning Mode (Alt 163)SET hive.exec.dynamic.partition.mode=nonstrict;Enables dynamic partitioning without static key.
165Set Dynamic Partitioning True (Alt 164)SET hive.exec.dynamic.partition=true;Activates dynamic partitioning execution.
166Set Max Dynamic Partitions (Alt 165)SET hive.exec.max.dynamic.partitions=10000;Increases max dynamic partition creation limit.
167Set Max Dynamic Partitions Per Node (Alt 166)SET hive.exec.max.dynamic.partitions.pernode=1000;Increases max dynamic partitions per mapper.
168Create Bucketed Table (Alt 167)CREATE TABLE enterprise_db.bucketed_users (id INT, name STRING) CLUSTERED BY (id) INTO 16 BUCKETS STORED AS ORC;Provisions bucketed table.
169Create Bucketed Sorted Table (Alt 168)CREATE TABLE enterprise_db.sorted_users (id INT, name STRING) CLUSTERED BY (id) SORTED BY (id) INTO 16 BUCKETS STORED AS ORC;Provisions bucketed and pre-sorted table.
170Set Enforce Bucketing (Alt 169)SET hive.enforce.bucketing=true;Enforces correct bucketing distribution on insert.
171Set Enforce Sorting (Alt 170)SET hive.enforce.sorting=true;Enforces correct row sorting on insert.
172Insert Overwrite Table (Alt 171)INSERT OVERWRITE TABLE enterprise_db.users SELECT * FROM enterprise_db.ext_users;Replaces table data rows.
173Insert Overwrite Partition (Alt 172)INSERT OVERWRITE TABLE enterprise_db.logs PARTITION (dt='2026-07-29') SELECT msg FROM staging;Replaces specific partition data.
174Alter Table Rename (Alt 173)ALTER TABLE enterprise_db.users RENAME TO enterprise_db.app_users;Renames table in metastore.
175Alter Table Add Columns (Alt 174)ALTER TABLE enterprise_db.users ADD COLUMNS (age INT, email STRING);Appends new columns to table schema.
176Alter Table Replace Columns (Alt 175)ALTER TABLE enterprise_db.users REPLACE COLUMNS (id INT, full_name STRING);Redefines table schema columns.
177Alter Table Change Column (Alt 176)ALTER TABLE enterprise_db.users CHANGE COLUMN name full_name STRING COMMENT 'Full user name';Renames and updates column comment.
178Alter Table Set Properties (Alt 177)ALTER TABLE enterprise_db.users SET TBLPROPERTIES ('comment'='Updated user table');Updates metastore table properties.
179Alter Table Set SerDe (Alt 178)ALTER TABLE enterprise_db.users SET SERDE 'org.apache.hadoop.hive.ql.io.parquet.serde.ParquetHiveSerDe';Updates table SerDe class.
180Alter Table Set Location (Alt 179)ALTER TABLE enterprise_db.users SET LOCATION '/apps/hive/warehouse/new_loc';Points table to new HDFS path.
181Show Partitions (Alt 180)SHOW PARTITIONS enterprise_db.logs;Lists all partition values for table.
182Show Create Table DDL (Alt 181)SHOW CREATE TABLE enterprise_db.users;Outputs exact DDL syntax used to build table.
183Show Functions (Alt 182)SHOW FUNCTIONS;Lists all built-in and custom UDF functions.
184Describe Function (Alt 183)DESCRIBE FUNCTION EXTENDED upper;Inspects parameters and usage of function.
185Create Temporary Function (Alt 184)CREATE TEMPORARY FUNCTION my_udf AS 'com.org.MyUDF';Provisions session-scoped custom UDF.
186Drop Temporary Function (Alt 185)DROP TEMPORARY FUNCTION IF EXISTS my_udf;Deletes session-scoped UDF.
187Create Permanent Function (Alt 186)CREATE FUNCTION enterprise_db.my_udf AS 'com.org.MyUDF' USING JAR 'hdfs:///udfs/my_udf.jar';Provisions metastore persistent UDF.
188Drop Permanent Function (Alt 187)DROP FUNCTION enterprise_db.my_udf;Deletes metastore persistent UDF.
189Create Index (Alt 188)CREATE INDEX idx_user_id ON TABLE enterprise_db.users (id) AS 'COMPACT' WITH DEFERRED REBUILD;Provisions legacy Hive index.
190Alter Index Rebuild (Alt 189)ALTER INDEX idx_user_id ON enterprise_db.users REBUILD;Builds legacy Hive index data.
191Drop Index (Alt 190)DROP INDEX IF EXISTS idx_user_id ON enterprise_db.users;Deletes legacy Hive index.
192Show Indexes (Alt 191)SHOW INDEXES ON enterprise_db.users;Lists all legacy indexes on table.
193Create Materialized View (Alt 192)CREATE MATERIALIZED VIEW enterprise_db.mv_summary STORED AS ORC AS SELECT dept, COUNT(*) FROM enterprise_db.users GROUP BY dept;Provisions auto-rewritten materialized view.
194Refresh Materialized View (Alt 193)ALTER MATERIALIZED VIEW enterprise_db.mv_summary REBUILD;Rebuilds materialized view dataset.
195Drop Materialized View (Alt 194)DROP MATERIALIZED VIEW enterprise_db.mv_summary;Deletes materialized view.
196Show Materialized Views (Alt 195)SHOW MATERIALIZED VIEWS IN enterprise_db;Lists all materialized views in database.
197Grant Role To User (Alt 196)GRANT ROLE data_analyst TO USER jlawrence;Assigns role to user account.
198Revoke Role From User (Alt 197)REVOKE ROLE data_analyst FROM USER jlawrence;Revokes role from user account.
199Drop Role (Alt 198)DROP ROLE data_analyst;Deletes security role.
200Show Roles (Alt 199)SHOW ROLES;Lists all security roles in metastore.

Apache Iceberg OLAP

Technical Architecture & Overview

Apache Iceberg is an open table format for huge analytic datasets. It provides full ACID transaction guarantees, schema evolution without side effects, partition evolution without rewriting tables, time travel querying, and fast metadata-driven filtering across compute engines (Spark, Trino, Flink, DuckDB, Snowflake).

Primary Use Cases: Open lakehouse data architectures, large-scale S3/ADLS/GCS analytics, high-frequency CDCupsert processing, time-travel auditing, and multi-engine data access.

Core Components: Iceberg Catalog (REST / Glue / Nessie / HMS), Metadata Files (JSON), Manifest Lists (AVRO), Manifest Files (AVRO), and Parquet/ORC Data Files.

Exhaustive 200-command administrative, diagnostic, and operational matrix for Apache Iceberg.

#Operation / ActionSpark SQL / PySpark SyntaxDescription
1Create Iceberg CatalogCREATE CATALOG spark_catalog TYPE iceberg WAREHOUSE 's3://my-bucket/iceberg';Provisions Iceberg table catalog.
2Create Iceberg TableCREATE TABLE spark_catalog.enterprise_db.users (id INT, name STRING, dept STRING) USING iceberg;Provisions open Iceberg table.
3Drop Iceberg TableDROP TABLE spark_catalog.enterprise_db.users;Deletes Iceberg table metadata and data.
4Insert RecordsINSERT INTO spark_catalog.enterprise_db.users VALUES (1, 'Alice', 'Engineering');Writes records into Iceberg table.
5Query Iceberg TableSELECT * FROM spark_catalog.enterprise_db.users;Reads data from Iceberg table.
6Describe Table HistorySELECT * FROM spark_catalog.enterprise_db.users.history;Inspects transactional metadata snapshots.
7Describe Table SnapshotsSELECT * FROM spark_catalog.enterprise_db.users.snapshots;Outputs snapshot audit history.
8Describe Table FilesSELECT * FROM spark_catalog.enterprise_db.users.files;Inspects physical data files.
9Time Travel QuerySELECT * FROM spark_catalog.enterprise_db.users TIMESTAMP AS OF '2026-07-01 10:00:00';Queries historical snapshot state.
10Rollback Table SnapshotCALL spark_catalog.system.rollback_to_snapshot('enterprise_db.users', 8492038420);Rolls back table state to snapshot ID.
11Expire Old SnapshotsCALL spark_catalog.system.expire_snapshots('enterprise_db.users', TIMESTAMP '2026-07-01 00:00:00');Removes expired snapshot metadata.
12Remove Orphan FilesCALL spark_catalog.system.remove_orphan_files('enterprise_db.users');Deletes unreferenced files from storage.
13Rewrite Data Files (Compact)CALL spark_catalog.system.rewrite_data_files('enterprise_db.users');Compacts small data files into larger blocks.
14Rewrite ManifestsCALL spark_catalog.system.rewrite_manifests('enterprise_db.users');Optimizes metadata manifest files.
15Add Table PartitionALTER TABLE spark_catalog.enterprise_db.users ADD PARTITION FIELD dept;Adds table partition transform.
16Drop Table PartitionALTER TABLE spark_catalog.enterprise_db.users DROP PARTITION FIELD dept;Removes partition transform.
17Add Table ColumnALTER TABLE spark_catalog.enterprise_db.users ADD COLUMN email STRING;Modifies table schema definition.
18Drop Table ColumnALTER TABLE spark_catalog.enterprise_db.users DROP COLUMN email;Removes table column.
19Rename Table ColumnALTER TABLE spark_catalog.enterprise_db.users RENAME COLUMN name TO full_name;Renames table column.
20Migrate Existing Table to IcebergCALL spark_catalog.system.migrate('spark_catalog.enterprise_db.legacy_users');Converts Hive/Parquet table to Iceberg.
21Snapshot Existing Table to IcebergCALL spark_catalog.system.snapshot('spark_catalog.enterprise_db.legacy_users', 's3://bucket/snapshot');Creates Iceberg snapshot without modifying legacy table.
22Fast Append FilesCALL spark_catalog.system.fast_append('enterprise_db.users', ARRAY('s3://bucket/data.parquet'));Quickly registers external Parquet file.
23Set Table PropertyALTER TABLE spark_catalog.enterprise_db.users SET TBLPROPERTIES ('write.format.default'='orc');Modifies Iceberg table configuration.
24Query Table Manifest EntriesSELECT * FROM spark_catalog.enterprise_db.users.manifests;Inspects manifest file metadata.
25Query Table PartitionsSELECT * FROM spark_catalog.enterprise_db.users.partitions;Inspects table partition boundaries.
26Cherry-pick SnapshotCALL spark_catalog.system.cherry_pick('enterprise_db.users', 123456789);Applies changes from another branch snapshot.
27Create BranchALTER TABLE spark_catalog.enterprise_db.users CREATE BRANCH audit_branch AS 123456789;Provisions isolated table branch.
28Create TagALTER TABLE spark_catalog.enterprise_db.users CREATE TAG release_v1 AS 123456789;Provisions immutable table tag.
29Inspect Table Metadata JSONDESCRIBE EXTENDED spark_catalog.enterprise_db.users;Outputs metadata file location.
30Drop Table BranchALTER TABLE spark_catalog.enterprise_db.users DROP BRANCH audit_branch;Removes isolated table branch.
31Drop Table TagALTER TABLE spark_catalog.enterprise_db.users DROP TAG release_v1;Removes immutable table tag.
32Set Branch Ref RetentionALTER TABLE spark_catalog.enterprise_db.users CREATE BRANCH dev_branch RETAIN 7 DAYS;Provisions branch with 7-day TTL.
33Replace Branch RefALTER TABLE spark_catalog.enterprise_db.users REPLACE BRANCH audit_branch AS 987654321;Updates branch head pointer.
34Query Branch DataSELECT * FROM spark_catalog.enterprise_db.users.branch_audit_branch;Queries data directly from branch ref.
35Query Tag DataSELECT * FROM spark_catalog.enterprise_db.users.tag_release_v1;Queries data directly from immutable tag ref.
36Time Travel Snapshot IDSELECT * FROM spark_catalog.enterprise_db.users VERSION AS OF 8492038420;Queries historical state by snapshot ID.
37Create Partition Transform BucketALTER TABLE spark_catalog.enterprise_db.users ADD PARTITION FIELD bucket(10, id);Adds 10-bucket partition transform.
38Create Partition Transform TruncateALTER TABLE spark_catalog.enterprise_db.users ADD PARTITION FIELD truncate(4, name);Adds prefix-4 string truncate partition.
39Create Partition Transform DayALTER TABLE spark_catalog.enterprise_db.users ADD PARTITION FIELD day(created_at);Adds daily time partition transform.
40Create Partition Transform HourALTER TABLE spark_catalog.enterprise_db.users ADD PARTITION FIELD hour(created_at);Adds hourly time partition transform.
41Create Partition Transform MonthALTER TABLE spark_catalog.enterprise_db.users ADD PARTITION FIELD month(created_at);Adds monthly time partition transform.
42Create Partition Transform YearALTER TABLE spark_catalog.enterprise_db.users ADD PARTITION FIELD year(created_at);Adds yearly time partition transform.
43Replace Partition FieldALTER TABLE spark_catalog.enterprise_db.users REPLACE PARTITION FIELD day(created_at) WITH month(created_at);Evolves partition layout without rewrite.
44Update Column Type (Promote)ALTER TABLE spark_catalog.enterprise_db.users ALTER COLUMN id TYPE BIGINT;Safely promotes INT column to BIGINT.
45Update Column CommentALTER TABLE spark_catalog.enterprise_db.users ALTER COLUMN name COMMENT 'Full user name';Updates column comment metadata.
46Update Column Order (First)ALTER TABLE spark_catalog.enterprise_db.users ALTER COLUMN id FIRST;Reorders column position to first.
47Update Column Order (After)ALTER TABLE spark_catalog.enterprise_db.users ALTER COLUMN email AFTER name;Reorders column position after name.
48Set Column Required (Not Null)ALTER TABLE spark_catalog.enterprise_db.users ALTER COLUMN id SET NOT NULL;Enforces non-null constraint on column.
49Drop Column Required (Nullable)ALTER TABLE spark_catalog.enterprise_db.users ALTER COLUMN id DROP NOT NULL;Allows NULL values in column.
50Create Nested Struct ColumnALTER TABLE spark_catalog.enterprise_db.users ADD COLUMN address STRUCT<street: STRING, zip: INT>;Appends nested struct column.
51Add Nested Field to StructALTER TABLE spark_catalog.enterprise_db.users ADD COLUMN address.city STRING;Appends field inside nested struct.
52Compact Data Files Strategy BinpackCALL spark_catalog.system.rewrite_data_files(table => 'enterprise_db.users', strategy => 'binpack');Compacts small files via binpack.
53Compact Data Files Strategy SortCALL spark_catalog.system.rewrite_data_files(table => 'enterprise_db.users', strategy => 'sort', sort_order => 'id ASC');Compacts and sorts data files.
54Compact Data Files Strategy ZOrderCALL spark_catalog.system.rewrite_data_files(table => 'enterprise_db.users', strategy => 'zorder', sort_order => 'id, dept');Compacts files using Z-Order curve.
55Set Write Parquet CompressionALTER TABLE spark_catalog.enterprise_db.users SET TBLPROPERTIES ('write.parquet.compression-codec'='zstd');Enforces ZSTD compression for Parquet.
56Set Write Target File SizeALTER TABLE spark_catalog.enterprise_db.users SET TBLPROPERTIES ('write.target-file-size-bytes'='268435456');Configures 256MB target data file size.
57Set Delete Mode Copy On WriteALTER TABLE spark_catalog.enterprise_db.users SET TBLPROPERTIES ('write.delete.mode'='copy-on-write');Enforces Copy-On-Write for DELETE.
58Set Delete Mode Merge On ReadALTER TABLE spark_catalog.enterprise_db.users SET TBLPROPERTIES ('write.delete.mode'='merge-on-read');Enforces Merge-On-Read for DELETE.
59Set Update Mode Merge On ReadALTER TABLE spark_catalog.enterprise_db.users SET TBLPROPERTIES ('write.update.mode'='merge-on-read');Enforces Merge-On-Read for UPDATE.
60Set Merge Mode Merge On ReadALTER TABLE spark_catalog.enterprise_db.users SET TBLPROPERTIES ('write.merge.mode'='merge-on-read');Enforces Merge-On-Read for MERGE INTO.
61Set Vectorized Read EnabledALTER TABLE spark_catalog.enterprise_db.users SET TBLPROPERTIES ('read.parquet.vectorization.enabled'='true');Activates vectorized Arrow reads.
62Set Manifest Target SizeALTER TABLE spark_catalog.enterprise_db.users SET TBLPROPERTIES ('commit.manifest.target-size-bytes'='8388608');Sets 8MB target size for manifest files.
63Set Manifest Min Count to MergeALTER TABLE spark_catalog.enterprise_db.users SET TBLPROPERTIES ('commit.manifest-min-merge-count'='10');Sets min manifests count before auto-merge.
64Set Snapshot Retain HoursALTER TABLE spark_catalog.enterprise_db.users SET TBLPROPERTIES ('history.expire.max-snapshot-age-ms'='604800000');Configures 7-day max snapshot retention.
65Set Snapshot Min CountALTER TABLE spark_catalog.enterprise_db.users SET TBLPROPERTIES ('history.expire.min-snapshots-to-keep'='10');Preserves at least 10 historical snapshots.
66Query Metadata Entries TableSELECT * FROM spark_catalog.enterprise_db.users.entries;Inspects active and deleted manifest entries.
67Query Metadata Refs TableSELECT * FROM spark_catalog.enterprise_db.users.refs;Lists active branches, tags, and snapshot pointers.
68Query Metadata All Data FilesSELECT * FROM spark_catalog.enterprise_db.users.all_data_files;Inspects data files across all historical snapshots.
69Query Metadata All ManifestsSELECT * FROM spark_catalog.enterprise_db.users.all_manifests;Inspects manifest files across all historical snapshots.
70Merge Into Iceberg TableMERGE INTO spark_catalog.enterprise_db.users t USING staging s ON t.id = s.id WHEN MATCHED THEN UPDATE SET t.name = s.name WHEN NOT MATCHED THEN INSERT (id, name, dept) VALUES (s.id, s.name, s.dept);Executes MERGE upsert operation.
71Update Iceberg Table RowsUPDATE spark_catalog.enterprise_db.users SET dept = 'R&D' WHERE id = 100;Modifies table row values.
72Delete Iceberg Table RowsDELETE FROM spark_catalog.enterprise_db.users WHERE id = 100;Deletes matching table rows.
73Set Current Snapshot RollbackCALL spark_catalog.system.set_current_snapshot('enterprise_db.users', 8492038420);Switches current active snapshot pointer.
74Ancestors Of SnapshotSELECT * FROM spark_catalog.enterprise_db.users.ancestors_of(8492038420);Inspects ancestor lineage of specific snapshot.
75Create REST CatalogCREATE CATALOG rest_catalog TYPE rest URI 'https://catalog.org/api/v1' WAREHOUSE 's3://bucket/iceberg';Provisions Iceberg REST catalog.
76Create Glue CatalogCREATE CATALOG glue_catalog TYPE glue WAREHOUSE 's3://bucket/iceberg';Provisions AWS Glue Iceberg catalog.
77Create Nessie CatalogCREATE CATALOG nessie_catalog TYPE nessie URI 'http://localhost:19120/api/v1' REF 'main' WAREHOUSE 's3://bucket/iceberg';Provisions Project Nessie git-like catalog.
78Create Hive Metastore CatalogCREATE CATALOG hive_catalog TYPE hive URI 'thrift://localhost:9083' WAREHOUSE 's3://bucket/iceberg';Provisions Hive Metastore Iceberg catalog.
79Create JDBC CatalogCREATE CATALOG jdbc_catalog TYPE jdbc URI 'jdbc:postgresql://localhost:5432/iceberg_catalog' WAREHOUSE 's3://bucket/iceberg';Provisions RDBMS JDBC Iceberg catalog.
80Show CatalogsSHOW CATALOGS;Lists all registered Iceberg catalogs.
81Show Schemas in CatalogSHOW SCHEMAS IN spark_catalog;Lists all database schemas in Iceberg catalog.
82Show Tables in SchemaSHOW TABLES IN spark_catalog.enterprise_db;Lists all Iceberg tables in schema.
83Create Schema in CatalogCREATE SCHEMA IF NOT EXISTS spark_catalog.enterprise_db;Provisions database schema namespace in catalog.
84Drop Schema in CatalogDROP SCHEMA spark_catalog.enterprise_db;Deletes database schema namespace from catalog.
85Show Table PropertiesSHOW TBLPROPERTIES spark_catalog.enterprise_db.users;Lists all configuration properties for table.
86Unset Table PropertyALTER TABLE spark_catalog.enterprise_db.users UNSET TBLPROPERTIES ('write.format.default');Removes configuration property from table.
87Create Table As Select (CTAS)CREATE TABLE spark_catalog.enterprise_db.users_summary AS SELECT dept, count(*) as cnt FROM spark_catalog.enterprise_db.users GROUP BY dept;Provisions new Iceberg table via query.
88Replace Table As Select (RTAS)CREATE OR REPLACE TABLE spark_catalog.enterprise_db.users_summary AS SELECT dept, count(*) as cnt FROM spark_catalog.enterprise_db.users GROUP BY dept;Atomically replaces table content and schema.
89Truncate Iceberg TableTRUNCATE TABLE spark_catalog.enterprise_db.users;Clears all data rows while maintaining table schema.
90Check Iceberg Library VersionSELECT iceberg_version();Outputs running Apache Iceberg library version string.
91Verify Storage Write PermissionsCALL spark_catalog.system.check_write('enterprise_db.users');Verifies write access to S3/HDFS deep storage.
92Check Manifest List Files CountSELECT count(*) FROM spark_catalog.enterprise_db.users.manifests;Counts total manifest files for current snapshot.
93Check Active Data Files CountSELECT count(*) FROM spark_catalog.enterprise_db.users.files;Counts total data files in current snapshot.
94Check Total Table RecordsSELECT sum(record_count) FROM spark_catalog.enterprise_db.users.files;Calculates total row count without scanning data files.
95Check Total Table File BytesSELECT sum(file_size_in_bytes) FROM spark_catalog.enterprise_db.users.files;Calculates total physical storage bytes without scanning files.
96Drop Branch SafelyALTER TABLE spark_catalog.enterprise_db.users DROP BRANCH IF EXISTS dev_branch;Safely drops branch reference if it exists.
97Drop Tag SafelyALTER TABLE spark_catalog.enterprise_db.users DROP TAG IF EXISTS release_v1;Safely drops tag reference if it exists.
98Show Active BranchesSELECT * FROM spark_catalog.enterprise_db.users.refs WHERE type = 'BRANCH';Lists all active branch references for table.
99Show Active TagsSELECT * FROM spark_catalog.enterprise_db.users.refs WHERE type = 'TAG';Lists all active tag references for table.
100Commit Table Metadata ManualCALL spark_catalog.system.commit('enterprise_db.users');Forces metadata commit to catalog.
101Check Table Current Snapshot IDSELECT current_snapshot_id FROM spark_catalog.enterprise_db.users.snapshots ORDER BY committed_at DESC LIMIT 1;Inspects current active snapshot ID.
102Check Table History Commit CountSELECT count(*) FROM spark_catalog.enterprise_db.users.history;Counts total committed snapshots in history.
103Check Table Snapshots CountSELECT count(*) FROM spark_catalog.enterprise_db.users.snapshots;Counts total active snapshots.
104Check Table Data Files CountSELECT count(*) FROM spark_catalog.enterprise_db.users.files;Counts total active physical Parquet data files.
105Check Table Total Storage Size BytesSELECT sum(file_size_in_bytes) FROM spark_catalog.enterprise_db.users.files;Calculates physical storage size of current snapshot.
106Check Table Total Row CountSELECT sum(record_count) FROM spark_catalog.enterprise_db.users.files;Calculates total row count without scanning data files.
107Check Table Manifest Files CountSELECT count(*) FROM spark_catalog.enterprise_db.users.manifests;Counts manifest files in current snapshot.
108Check Table Partition Specs ListSELECT * FROM spark_catalog.enterprise_db.users.partitions;Inspects active partition spec boundaries.
109Check Table Refs ListSELECT * FROM spark_catalog.enterprise_db.users.refs;Lists active branch and tag references.
110Check Table Entries ListSELECT * FROM spark_catalog.enterprise_db.users.entries;Lists manifest entry actions (ADD/DELETE).
111Check Table All Data FilesSELECT * FROM spark_catalog.enterprise_db.users.all_data_files;Lists data files across all historical snapshots.
112Check Table All ManifestsSELECT * FROM spark_catalog.enterprise_db.users.all_manifests;Lists manifest files across all historical snapshots.
113Check Table Properties ListSHOW TBLPROPERTIES spark_catalog.enterprise_db.users;Lists all table configuration properties.
114Set Table Format ParquetALTER TABLE spark_catalog.enterprise_db.users SET TBLPROPERTIES ('write.format.default'='parquet');Enforces Parquet write format.
115Set Table Format ORCALTER TABLE spark_catalog.enterprise_db.users SET TBLPROPERTIES ('write.format.default'='orc');Enforces ORC write format.
116Set Table Format AvroALTER TABLE spark_catalog.enterprise_db.users SET TBLPROPERTIES ('write.format.default'='avro');Enforces Avro write format.
117Set Parquet Compression ZSTDALTER TABLE spark_catalog.enterprise_db.users SET TBLPROPERTIES ('write.parquet.compression-codec'='zstd');Sets ZSTD compression codec.
118Set Parquet Compression SnappyALTER TABLE spark_catalog.enterprise_db.users SET TBLPROPERTIES ('write.parquet.compression-codec'='snappy');Sets Snappy compression codec.
119Set Target File Size 256MBALTER TABLE spark_catalog.enterprise_db.users SET TBLPROPERTIES ('write.target-file-size-bytes'='268435456');Sets 256MB target data file size.
120Set Delete Mode Copy On WriteALTER TABLE spark_catalog.enterprise_db.users SET TBLPROPERTIES ('write.delete.mode'='copy-on-write');Enforces Copy-On-Write for DELETE.
121Set Delete Mode Merge On ReadALTER TABLE spark_catalog.enterprise_db.users SET TBLPROPERTIES ('write.delete.mode'='merge-on-read');Enforces Merge-On-Read for DELETE.
122Set Update Mode Copy On WriteALTER TABLE spark_catalog.enterprise_db.users SET TBLPROPERTIES ('write.update.mode'='copy-on-write');Enforces Copy-On-Write for UPDATE.
123Set Update Mode Merge On ReadALTER TABLE spark_catalog.enterprise_db.users SET TBLPROPERTIES ('write.update.mode'='merge-on-read');Enforces Merge-On-Read for UPDATE.
124Set Merge Mode Copy On WriteALTER TABLE spark_catalog.enterprise_db.users SET TBLPROPERTIES ('write.merge.mode'='copy-on-write');Enforces Copy-On-Write for MERGE INTO.
125Set Merge Mode Merge On ReadALTER TABLE spark_catalog.enterprise_db.users SET TBLPROPERTIES ('write.merge.mode'='merge-on-read');Enforces Merge-On-Read for MERGE INTO.
126Set Snapshot Retention Max AgeALTER TABLE spark_catalog.enterprise_db.users SET TBLPROPERTIES ('history.expire.max-snapshot-age-ms'='604800000');Sets 7-day max snapshot retention.
127Set Snapshot Retention Min CountALTER TABLE spark_catalog.enterprise_db.users SET TBLPROPERTIES ('history.expire.min-snapshots-to-keep'='10');Preserves at least 10 historical snapshots.
128Check Iceberg Library VersionSELECT iceberg_version();Outputs running Iceberg library version string.
129Check Ping Connection QuerySELECT 1;Simple connection ping test returning 1.
130Check Current User AccountSELECT current_user();Outputs active execution user name.
131Create Iceberg Catalog (Alt 130)CREATE CATALOG spark_catalog TYPE iceberg WAREHOUSE 's3://my-bucket/iceberg';Provisions Iceberg table catalog.
132Create Iceberg Table (Alt 131)CREATE TABLE spark_catalog.enterprise_db.users (id INT, name STRING, dept STRING) USING iceberg;Provisions open Iceberg table.
133Drop Iceberg Table (Alt 132)DROP TABLE spark_catalog.enterprise_db.users;Deletes Iceberg table metadata and data.
134Insert Records (Alt 133)INSERT INTO spark_catalog.enterprise_db.users VALUES (1, 'Alice', 'Engineering');Writes records into Iceberg table.
135Query Iceberg Table (Alt 134)SELECT * FROM spark_catalog.enterprise_db.users;Reads data from Iceberg table.
136Describe Table History (Alt 135)SELECT * FROM spark_catalog.enterprise_db.users.history;Inspects transactional metadata snapshots.
137Describe Table Snapshots (Alt 136)SELECT * FROM spark_catalog.enterprise_db.users.snapshots;Outputs snapshot audit history.
138Describe Table Files (Alt 137)SELECT * FROM spark_catalog.enterprise_db.users.files;Inspects physical data files.
139Time Travel Query (Alt 138)SELECT * FROM spark_catalog.enterprise_db.users TIMESTAMP AS OF '2026-07-01 10:00:00';Queries historical snapshot state.
140Rollback Table Snapshot (Alt 139)CALL spark_catalog.system.rollback_to_snapshot('enterprise_db.users', 8492038420);Rolls back table state to snapshot ID.
141Expire Old Snapshots (Alt 140)CALL spark_catalog.system.expire_snapshots('enterprise_db.users', TIMESTAMP '2026-07-01 00:00:00');Removes expired snapshot metadata.
142Remove Orphan Files (Alt 141)CALL spark_catalog.system.remove_orphan_files('enterprise_db.users');Deletes unreferenced files from storage.
143Rewrite Data Files (Compact) (Alt 142)CALL spark_catalog.system.rewrite_data_files('enterprise_db.users');Compacts small data files into larger blocks.
144Rewrite Manifests (Alt 143)CALL spark_catalog.system.rewrite_manifests('enterprise_db.users');Optimizes metadata manifest files.
145Add Table Partition (Alt 144)ALTER TABLE spark_catalog.enterprise_db.users ADD PARTITION FIELD dept;Adds table partition transform.
146Drop Table Partition (Alt 145)ALTER TABLE spark_catalog.enterprise_db.users DROP PARTITION FIELD dept;Removes partition transform.
147Add Table Column (Alt 146)ALTER TABLE spark_catalog.enterprise_db.users ADD COLUMN email STRING;Modifies table schema definition.
148Drop Table Column (Alt 147)ALTER TABLE spark_catalog.enterprise_db.users DROP COLUMN email;Removes table column.
149Rename Table Column (Alt 148)ALTER TABLE spark_catalog.enterprise_db.users RENAME COLUMN name TO full_name;Renames table column.
150Migrate Existing Table to Iceberg (Alt 149)CALL spark_catalog.system.migrate('spark_catalog.enterprise_db.legacy_users');Converts Hive/Parquet table to Iceberg.
151Snapshot Existing Table to Iceberg (Alt 150)CALL spark_catalog.system.snapshot('spark_catalog.enterprise_db.legacy_users', 's3://bucket/snapshot');Creates Iceberg snapshot without modifying legacy table.
152Fast Append Files (Alt 151)CALL spark_catalog.system.fast_append('enterprise_db.users', ARRAY('s3://bucket/data.parquet'));Quickly registers external Parquet file.
153Set Table Property (Alt 152)ALTER TABLE spark_catalog.enterprise_db.users SET TBLPROPERTIES ('write.format.default'='orc');Modifies Iceberg table configuration.
154Query Table Manifest Entries (Alt 153)SELECT * FROM spark_catalog.enterprise_db.users.manifests;Inspects manifest file metadata.
155Query Table Partitions (Alt 154)SELECT * FROM spark_catalog.enterprise_db.users.partitions;Inspects table partition boundaries.
156Cherry-pick Snapshot (Alt 155)CALL spark_catalog.system.cherry_pick('enterprise_db.users', 123456789);Applies changes from another branch snapshot.
157Create Branch (Alt 156)ALTER TABLE spark_catalog.enterprise_db.users CREATE BRANCH audit_branch AS 123456789;Provisions isolated table branch.
158Create Tag (Alt 157)ALTER TABLE spark_catalog.enterprise_db.users CREATE TAG release_v1 AS 123456789;Provisions immutable table tag.
159Inspect Table Metadata JSON (Alt 158)DESCRIBE EXTENDED spark_catalog.enterprise_db.users;Outputs metadata file location.
160Drop Table Branch (Alt 159)ALTER TABLE spark_catalog.enterprise_db.users DROP BRANCH audit_branch;Removes isolated table branch.
161Drop Table Tag (Alt 160)ALTER TABLE spark_catalog.enterprise_db.users DROP TAG release_v1;Removes immutable table tag.
162Set Branch Ref Retention (Alt 161)ALTER TABLE spark_catalog.enterprise_db.users CREATE BRANCH dev_branch RETAIN 7 DAYS;Provisions branch with 7-day TTL.
163Replace Branch Ref (Alt 162)ALTER TABLE spark_catalog.enterprise_db.users REPLACE BRANCH audit_branch AS 987654321;Updates branch head pointer.
164Query Branch Data (Alt 163)SELECT * FROM spark_catalog.enterprise_db.users.branch_audit_branch;Queries data directly from branch ref.
165Query Tag Data (Alt 164)SELECT * FROM spark_catalog.enterprise_db.users.tag_release_v1;Queries data directly from immutable tag ref.
166Time Travel Snapshot ID (Alt 165)SELECT * FROM spark_catalog.enterprise_db.users VERSION AS OF 8492038420;Queries historical state by snapshot ID.
167Create Partition Transform Bucket (Alt 166)ALTER TABLE spark_catalog.enterprise_db.users ADD PARTITION FIELD bucket(10, id);Adds 10-bucket partition transform.
168Create Partition Transform Truncate (Alt 167)ALTER TABLE spark_catalog.enterprise_db.users ADD PARTITION FIELD truncate(4, name);Adds prefix-4 string truncate partition.
169Create Partition Transform Day (Alt 168)ALTER TABLE spark_catalog.enterprise_db.users ADD PARTITION FIELD day(created_at);Adds daily time partition transform.
170Create Partition Transform Hour (Alt 169)ALTER TABLE spark_catalog.enterprise_db.users ADD PARTITION FIELD hour(created_at);Adds hourly time partition transform.
171Create Partition Transform Month (Alt 170)ALTER TABLE spark_catalog.enterprise_db.users ADD PARTITION FIELD month(created_at);Adds monthly time partition transform.
172Create Partition Transform Year (Alt 171)ALTER TABLE spark_catalog.enterprise_db.users ADD PARTITION FIELD year(created_at);Adds yearly time partition transform.
173Replace Partition Field (Alt 172)ALTER TABLE spark_catalog.enterprise_db.users REPLACE PARTITION FIELD day(created_at) WITH month(created_at);Evolves partition layout without rewrite.
174Update Column Type (Promote) (Alt 173)ALTER TABLE spark_catalog.enterprise_db.users ALTER COLUMN id TYPE BIGINT;Safely promotes INT column to BIGINT.
175Update Column Comment (Alt 174)ALTER TABLE spark_catalog.enterprise_db.users ALTER COLUMN name COMMENT 'Full user name';Updates column comment metadata.
176Update Column Order (First) (Alt 175)ALTER TABLE spark_catalog.enterprise_db.users ALTER COLUMN id FIRST;Reorders column position to first.
177Update Column Order (After) (Alt 176)ALTER TABLE spark_catalog.enterprise_db.users ALTER COLUMN email AFTER name;Reorders column position after name.
178Set Column Required (Not Null) (Alt 177)ALTER TABLE spark_catalog.enterprise_db.users ALTER COLUMN id SET NOT NULL;Enforces non-null constraint on column.
179Drop Column Required (Nullable) (Alt 178)ALTER TABLE spark_catalog.enterprise_db.users ALTER COLUMN id DROP NOT NULL;Allows NULL values in column.
180Create Nested Struct Column (Alt 179)ALTER TABLE spark_catalog.enterprise_db.users ADD COLUMN address STRUCT<street: STRING, zip: INT>;Appends nested struct column.
181Add Nested Field to Struct (Alt 180)ALTER TABLE spark_catalog.enterprise_db.users ADD COLUMN address.city STRING;Appends field inside nested struct.
182Compact Data Files Strategy Binpack (Alt 181)CALL spark_catalog.system.rewrite_data_files(table => 'enterprise_db.users', strategy => 'binpack');Compacts small files via binpack.
183Compact Data Files Strategy Sort (Alt 182)CALL spark_catalog.system.rewrite_data_files(table => 'enterprise_db.users', strategy => 'sort', sort_order => 'id ASC');Compacts and sorts data files.
184Compact Data Files Strategy ZOrder (Alt 183)CALL spark_catalog.system.rewrite_data_files(table => 'enterprise_db.users', strategy => 'zorder', sort_order => 'id, dept');Compacts files using Z-Order curve.
185Set Write Parquet Compression (Alt 184)ALTER TABLE spark_catalog.enterprise_db.users SET TBLPROPERTIES ('write.parquet.compression-codec'='zstd');Enforces ZSTD compression for Parquet.
186Set Write Target File Size (Alt 185)ALTER TABLE spark_catalog.enterprise_db.users SET TBLPROPERTIES ('write.target-file-size-bytes'='268435456');Configures 256MB target data file size.
187Set Delete Mode Copy On Write (Alt 186)ALTER TABLE spark_catalog.enterprise_db.users SET TBLPROPERTIES ('write.delete.mode'='copy-on-write');Enforces Copy-On-Write for DELETE.
188Set Delete Mode Merge On Read (Alt 187)ALTER TABLE spark_catalog.enterprise_db.users SET TBLPROPERTIES ('write.delete.mode'='merge-on-read');Enforces Merge-On-Read for DELETE.
189Set Update Mode Merge On Read (Alt 188)ALTER TABLE spark_catalog.enterprise_db.users SET TBLPROPERTIES ('write.update.mode'='merge-on-read');Enforces Merge-On-Read for UPDATE.
190Set Merge Mode Merge On Read (Alt 189)ALTER TABLE spark_catalog.enterprise_db.users SET TBLPROPERTIES ('write.merge.mode'='merge-on-read');Enforces Merge-On-Read for MERGE INTO.
191Set Vectorized Read Enabled (Alt 190)ALTER TABLE spark_catalog.enterprise_db.users SET TBLPROPERTIES ('read.parquet.vectorization.enabled'='true');Activates vectorized Arrow reads.
192Set Manifest Target Size (Alt 191)ALTER TABLE spark_catalog.enterprise_db.users SET TBLPROPERTIES ('commit.manifest.target-size-bytes'='8388608');Sets 8MB target size for manifest files.
193Set Manifest Min Count to Merge (Alt 192)ALTER TABLE spark_catalog.enterprise_db.users SET TBLPROPERTIES ('commit.manifest-min-merge-count'='10');Sets min manifests count before auto-merge.
194Set Snapshot Retain Hours (Alt 193)ALTER TABLE spark_catalog.enterprise_db.users SET TBLPROPERTIES ('history.expire.max-snapshot-age-ms'='604800000');Configures 7-day max snapshot retention.
195Set Snapshot Min Count (Alt 194)ALTER TABLE spark_catalog.enterprise_db.users SET TBLPROPERTIES ('history.expire.min-snapshots-to-keep'='10');Preserves at least 10 historical snapshots.
196Query Metadata Entries Table (Alt 195)SELECT * FROM spark_catalog.enterprise_db.users.entries;Inspects active and deleted manifest entries.
197Query Metadata Refs Table (Alt 196)SELECT * FROM spark_catalog.enterprise_db.users.refs;Lists active branches, tags, and snapshot pointers.
198Query Metadata All Data Files (Alt 197)SELECT * FROM spark_catalog.enterprise_db.users.all_data_files;Inspects data files across all historical snapshots.
199Query Metadata All Manifests (Alt 198)SELECT * FROM spark_catalog.enterprise_db.users.all_manifests;Inspects manifest files across all historical snapshots.
200Merge Into Iceberg Table (Alt 199)MERGE INTO spark_catalog.enterprise_db.users t USING staging s ON t.id = s.id WHEN MATCHED THEN UPDATE SET t.name = s.name WHEN NOT MATCHED THEN INSERT (id, name, dept) VALUES (s.id, s.name, s.dept);Executes MERGE upsert operation.

MongoDB Enterprise NOSQL

Technical Architecture & Overview

MongoDB Enterprise is a document-oriented NoSQL database that stores data in flexible, schema-validated BSON documents. It features distributed replica sets for high availability, automatic sharding for horizontal scale-out, multi-document ACID transactions, and a pipeline-based aggregation framework.

Primary Use Cases: Real-time operational data stores, content management, mobile app backends, customer 360 view stores, and flexible schema catalog management.

Core Components: mongod daemon, WiredTiger Storage Engine, Replica Set Elections (Raft variant), Shard Routers (mongos), Config Servers, and mongosh CLI.

Exhaustive 200-command administrative, diagnostic, and operational matrix for MongoDB Enterprise.

#Operation / Actionmongosh CLI / JS SyntaxDescription
1Open Shellmongosh 'mongodb://localhost:27017/enterprise_db'Opens administrative MongoDB shell.
2Start Servicesudo systemctl start mongodBoots MongoDB server daemon.
3Stop Servicesudo systemctl stop mongodStops MongoDB server daemon.
4Create Backup (mongodump)mongodump --uri="mongodb://localhost:27017/enterprise_db" --out=/backupExports database BSON binary backup.
5Restore Backup (mongorestore)mongorestore --uri="mongodb://localhost:27017/enterprise_db" /backup/enterprise_dbRestores database backup BSON files.
6Show Databasesshow dbsLists all database instances.
7Switch Databaseuse enterprise_dbSwitches active database context.
8Show Collectionsshow collectionsLists collections in database.
9Insert Documentdb.users.insertOne({ name: "Alice", role: "Admin", active: true })Writes JSON document into collection.
10Find Documentsdb.users.find({ active: true }).pretty()Queries documents from collection.
11Update Documentdb.users.updateOne({ name: "Alice" }, { $set: { role: "SuperAdmin" } })Modifies existing document records.
12Delete Documentdb.users.deleteOne({ name: "Alice" })Removes document from collection.
13Create Indexdb.users.createIndex({ email: 1 }, { unique: true })Provisions collection index.
14Drop Indexdb.users.dropIndex("email_1")Removes collection index.
15Run Aggregation Pipelinedb.users.aggregate([ { $match: { active: true } }, { $group: { _id: "$role", count: { $sum: 1 } } } ])Executes analytical aggregation pipeline.
16Explain Execution Plandb.users.find({ email: "test@org.com" }).explain("executionStats")Evaluates query index usage.
17Check Replica Set Statusrs.status()Inspects replica set high availability.
18Initialize Replica Setrs.initiate()Bootstraps replica set cluster.
19Add Replica Memberrs.add("mongodb-node2:27017")Adds secondary node to replica set.
20Create Database Userdb.createUser({ user: "appUser", pwd: "SecurePass!", roles: [ { role: "readWrite", db: "enterprise_db" } ] })Provisions database user account.
21Authenticate Userdb.auth("appUser", "SecurePass!")Authenticates user session.
22Check Current Operationsdb.currentOp()Inspects running database queries.
23Kill Operationdb.killOp(12345)Terminates hung query operation.
24Check Storage Statsdb.users.stats()Calculates collection storage sizes.
25Compact Collectiondb.runCommand({ compact: "users" })Defragments collection storage space.
26Enable Profilingdb.setProfilingLevel(1, { slowms: 100 })Activates slow query profiler.
27View Profiler Logsdb.system.profile.find().sort({ millis: -1 }).limit(5)Pulls slow query telemetry.
28Create Text Indexdb.users.createIndex({ bio: "text" })Provisions full-text search index.
29Execute Text Searchdb.users.find({ $text: { $search: "database architect" } })Performs full-text document search.
30Exit ShellexitCloses interactive shell.
31Remove Replica Memberrs.remove("mongodb-node2:27017")Removes secondary node from replica set.
32Step Down Primaryrs.stepDown(60)Forces primary node to step down to secondary for 60 seconds.
33Freeze Replica Memberrs.freeze(120)Prevents secondary node from becoming primary for 2 minutes.
34Reconfig Replica Setrs.reconfig(cfg)Applies new configuration to replica set.
35Get Replica Set Configrs.conf()Inspects current replica set configuration.
36Check Oplog Sizedb.getReplicationInfo()Inspects oplog size and replication window capacity.
37Check Replica Lagdb.printSecondaryReplicationInfo()Pulls replication delay status across secondary nodes.
38Enable Sharding on Databasesh.enableSharding("enterprise_db")Enables sharding on database in cluster.
39Shard Collectionsh.shardCollection("enterprise_db.users", { user_id: "hashed" })Shards collection across cluster using hashed key.
40Add Shard to Clustersh.addShard("rs1/mongodb-shard1:27017")Adds new replica set shard to cluster.
41Check Shard Statussh.status()Inspects sharding topology, chunks, and balancing.
42Enable Balancersh.startBalancer()Activates automated chunk balancing across shards.
43Disable Balancersh.stopBalancer()Deactivates chunk balancing during bulk migration.
44Check Balancer Statesh.getBalancerState()Inspects whether chunk balancer is enabled.
45Set Balancer Windowdb.settings.updateOne({ _id: "balancer" }, { $set: { activeWindow: { start: "00:00", stop: "06:00" } } }, { upsert: true })Restricts chunk balancing to off-peak hours.
46Move Chunk Manuallysh.moveChunk("enterprise_db.users", { user_id: 1000 }, "shard2")Manually migrates chunk to target shard.
47Split Chunk Manuallysh.splitAt("enterprise_db.users", { user_id: 5000 })Splits large chunk at specified split key.
48Create Compound Indexdb.users.createIndex({ role: 1, created_at: -1 })Provisions compound index for sorting and filtering.
49Create Partial Indexdb.users.createIndex({ email: 1 }, { partialFilterExpression: { active: true } })Provisions index for active documents only.
50Create Sparse Indexdb.users.createIndex({ phone: 1 }, { sparse: true })Provisions index indexing documents containing field.
51Create TTL Indexdb.sessions.createIndex({ lastAccess: 1 }, { expireAfterSeconds: 3600 })Provisions auto-expiring document index.
52Create 2d Sphere Indexdb.locations.createIndex({ loc: "2dsphere" })Provisions geospatial index for GeoJSON points.
53Create Wildcard Indexdb.products.createIndex({ "attributes.$": 1 })Provisions index on arbitrary sub-document fields.
54List Indexesdb.users.getIndexes()Lists all indexes created on collection.
55Reindex Collectiondb.users.reIndex()Rebuilds all indexes on collection.
56Hide Indexdb.users.hideIndex("email_1")Hides index from query planner without dropping.
57Unhide Indexdb.users.unhideIndex("email_1")Restores hidden index to query planner.
58Create Clustered Collectiondb.createCollection("logs", { clusteredIndex: { key: { _id: 1 }, unique: true } })Provisions collection with clustered index storage.
59Create Capped Collectiondb.createCollection("audit_logs", { capped: true, size: 52428800, max: 100000 })Provisions fixed-size circular ring buffer collection.
60Create Time Series Collectiondb.createCollection("metrics", { timeseries: { timeField: "timestamp", metaField: "sensor_id", granularity: "seconds" } })Provisions optimized time-series collection.
61Create Viewdb.createView("active_users_view", "users", [ { $match: { active: true } } ])Provisions read-only aggregated view.
62Drop Viewdb.active_users_view.drop()Deletes read-only view.
63Insert Many Documentsdb.users.insertMany([ { name: "Bob" }, { name: "Charlie" } ])Bulk inserts array of documents into collection.
64Update Many Documentsdb.users.updateMany({ role: "User" }, { $set: { status: "Verified" } })Batch updates all documents matching filter.
65Delete Many Documentsdb.users.deleteMany({ active: false })Batch deletes all documents matching filter.
66Replace Documentdb.users.replaceOne({ _id: ObjectId("60d5ec...") }, { name: "New Alice", role: "Admin" })Replaces entire document payload keeping _id.
67Find One and Updatedb.users.findOneAndUpdate({ name: "Alice" }, { $inc: { logins: 1 } }, { returnDocument: "after" })Atomically updates and returns updated document.
68Find One and Deletedb.users.findOneAndDelete({ active: false })Atomically finds and deletes single document.
69Bulk Write Operationsdb.users.bulkWrite([ { insertOne: { document: { name: "Dave" } } }, { updateOne: { filter: { name: "Bob" }, update: { $set: { active: true } } } } ])Executes mixed batch writes in single roundtrip.
70Count Documentsdb.users.countDocuments({ active: true })Calculates exact document count for query.
71Estimated Document Countdb.users.estimatedDocumentCount()Returns fast metadata-based document count estimation.
72Distinct Valuesdb.users.distinct("role")Fetches array of unique values for field.
73Create User Roledb.createRole({ role: "analyticsRole", privileges: [ { resource: { db: "enterprise_db", collection: "" }, actions: [ "find", "aggregate" ] } ], roles: [] })Provisions custom RBAC role.
74Drop User Roledb.dropRole("analyticsRole")Deletes custom RBAC role.
75Grant Roles To Userdb.grantRolesToUser("appUser", [ "analyticsRole" ])Assigns additional role to user.
76Revoke Roles From Userdb.revokeRolesFromUser("appUser", [ "analyticsRole" ])Revokes role from user.
77Change User Passworddb.changeUserPassword("appUser", "NewSecurePass123!")Updates database login password.
78Drop Userdb.dropUser("appUser")Deletes database user account.
79Show Usersdb.getUsers()Lists all user accounts in active database.
80Show Rolesdb.getRoles({ showBuiltinRoles: true })Lists all custom and built-in roles.
81Check Database Statsdb.stats()Pulls database size, collection count, and index size.
82Check Host Infodb.hostInfo()Inspects OS, CPU cores, and system memory.
83Check Server Statusdb.serverStatus()Pulls memory, connection, and opcounter metrics.
84Check Server Status Memorydb.serverStatus().memInspects resident, virtual, and mapped memory RAM.
85Check Server Status Connectionsdb.serverStatus().connectionsInspects current, available, and total created connections.
86Check WiredTiger Cache Statsdb.serverStatus().wiredTiger.cacheInspects WiredTiger cache dirty bytes and eviction stats.
87Rotate Log Files via Commanddb.adminCommand({ logRotate: 1 })Forces rotation of MongoDB server log files.
88Set Parameter Sessiondb.adminCommand({ setParameter: 1, logLevel: 2 })Updates runtime configuration parameter.
89Get Parameter Sessiondb.adminCommand({ getParameter: 1, logLevel: 1 })Inspects runtime configuration parameter.
90Shutdown Server via Commanddb.adminCommand({ shutdown: 1 })Gracefully shuts down MongoDB server daemon.
91Create Client Client Sessionsession = db.getMongo().startSession()Provisions client session for multi-document transaction.
92Start Transactionsession.startTransaction()Starts ACID multi-document transaction.
93Commit Transactionsession.commitTransaction()Finalizes multi-document transaction.
94Abort Transactionsession.abortTransaction()Aborts multi-document transaction.
95Check Transaction Metricsdb.serverStatus().transactionsMonitors active, committed, and aborted transactions.
96Check Free Disk Spacedb.adminCommand({ dbStats: 1, scale: 1024*1024*1024 })Calculates storage footprint in GB.
97Check Index Usage Metricsdb.users.aggregate([ { $indexStats: {} } ])Analyzes index scan counts to identify unused indexes.
98Check Plan Cache Listdb.users.getPlanCache().list()Lists cached query plans for collection.
99Clear Plan Cachedb.users.getPlanCache().clear()Purges cached query execution plans.
100Validate Collectiondb.users.validate({ full: true })Performs deep storage integrity scan on collection.
101Open Shell Option 1mongosh 'mongodb://localhost:27017/enterprise_db'Opens administrative MongoDB shell.
102Start Service Option 2sudo systemctl start mongodBoots MongoDB server daemon.
103Stop Service Option 3sudo systemctl stop mongodStops MongoDB server daemon.
104Create Backup (mongodump) Option 4mongodump --uri="mongodb://localhost:27017/enterprise_db" --out=/backupExports database BSON binary backup.
105Restore Backup (mongorestore) Option 5mongorestore --uri="mongodb://localhost:27017/enterprise_db" /backup/enterprise_dbRestores database backup BSON files.
106Show Databases Option 6show dbsLists all database instances.
107Switch Database Option 7use enterprise_dbSwitches active database context.
108Show Collections Option 8show collectionsLists collections in database.
109Insert Document Option 9db.users.insertOne({ name: "Alice", role: "Admin", active: true })Writes JSON document into collection.
110Find Documents Option 10db.users.find({ active: true }).pretty()Queries documents from collection.
111Update Document Option 11db.users.updateOne({ name: "Alice" }, { $set: { role: "SuperAdmin" } })Modifies existing document records.
112Delete Document Option 12db.users.deleteOne({ name: "Alice" })Removes document from collection.
113Create Index Option 13db.users.createIndex({ email: 1 }, { unique: true })Provisions collection index.
114Drop Index Option 14db.users.dropIndex("email_1")Removes collection index.
115Run Aggregation Pipeline Option 15db.users.aggregate([ { $match: { active: true } }, { $group: { _id: "$role", count: { $sum: 1 } } } ])Executes analytical aggregation pipeline.
116Explain Execution Plan Option 16db.users.find({ email: "test@org.com" }).explain("executionStats")Evaluates query index usage.
117Check Replica Set Status Option 17rs.status()Inspects replica set high availability.
118Initialize Replica Set Option 18rs.initiate()Bootstraps replica set cluster.
119Add Replica Member Option 19rs.add("mongodb-node2:27017")Adds secondary node to replica set.
120Create Database User Option 20db.createUser({ user: "appUser", pwd: "SecurePass!", roles: [ { role: "readWrite", db: "enterprise_db" } ] })Provisions database user account.
121Authenticate User Option 21db.auth("appUser", "SecurePass!")Authenticates user session.
122Check Current Operations Option 22db.currentOp()Inspects running database queries.
123Kill Operation Option 23db.killOp(12345)Terminates hung query operation.
124Check Storage Stats Option 24db.users.stats()Calculates collection storage sizes.
125Compact Collection Option 25db.runCommand({ compact: "users" })Defragments collection storage space.
126Enable Profiling Option 26db.setProfilingLevel(1, { slowms: 100 })Activates slow query profiler.
127View Profiler Logs Option 27db.system.profile.find().sort({ millis: -1 }).limit(5)Pulls slow query telemetry.
128Create Text Index Option 28db.users.createIndex({ bio: "text" })Provisions full-text search index.
129Execute Text Search Option 29db.users.find({ $text: { $search: "database architect" } })Performs full-text document search.
130Exit Shell Option 30exitCloses interactive shell.
131Remove Replica Member Option 31rs.remove("mongodb-node2:27017")Removes secondary node from replica set.
132Step Down Primary Option 32rs.stepDown(60)Forces primary node to step down to secondary for 60 seconds.
133Freeze Replica Member Option 33rs.freeze(120)Prevents secondary node from becoming primary for 2 minutes.
134Reconfig Replica Set Option 34rs.reconfig(cfg)Applies new configuration to replica set.
135Get Replica Set Config Option 35rs.conf()Inspects current replica set configuration.
136Check Oplog Size Option 36db.getReplicationInfo()Inspects oplog size and replication window capacity.
137Check Replica Lag Option 37db.printSecondaryReplicationInfo()Pulls replication delay status across secondary nodes.
138Enable Sharding on Database Option 38sh.enableSharding("enterprise_db")Enables sharding on database in cluster.
139Shard Collection Option 39sh.shardCollection("enterprise_db.users", { user_id: "hashed" })Shards collection across cluster using hashed key.
140Add Shard to Cluster Option 40sh.addShard("rs1/mongodb-shard1:27017")Adds new replica set shard to cluster.
141Check Shard Status Option 41sh.status()Inspects sharding topology, chunks, and balancing.
142Enable Balancer Option 42sh.startBalancer()Activates automated chunk balancing across shards.
143Disable Balancer Option 43sh.stopBalancer()Deactivates chunk balancing during bulk migration.
144Check Balancer State Option 44sh.getBalancerState()Inspects whether chunk balancer is enabled.
145Set Balancer Window Option 45db.settings.updateOne({ _id: "balancer" }, { $set: { activeWindow: { start: "00:00", stop: "06:00" } } }, { upsert: true })Restricts chunk balancing to off-peak hours.
146Move Chunk Manually Option 46sh.moveChunk("enterprise_db.users", { user_id: 1000 }, "shard2")Manually migrates chunk to target shard.
147Split Chunk Manually Option 47sh.splitAt("enterprise_db.users", { user_id: 5000 })Splits large chunk at specified split key.
148Create Compound Index Option 48db.users.createIndex({ role: 1, created_at: -1 })Provisions compound index for sorting and filtering.
149Create Partial Index Option 49db.users.createIndex({ email: 1 }, { partialFilterExpression: { active: true } })Provisions index for active documents only.
150Create Sparse Index Option 50db.users.createIndex({ phone: 1 }, { sparse: true })Provisions index indexing documents containing field.
151Create TTL Index Option 51db.sessions.createIndex({ lastAccess: 1 }, { expireAfterSeconds: 3600 })Provisions auto-expiring document index.
152Create 2d Sphere Index Option 52db.locations.createIndex({ loc: "2dsphere" })Provisions geospatial index for GeoJSON points.
153Create Wildcard Index Option 53db.products.createIndex({ "attributes.$": 1 })Provisions index on arbitrary sub-document fields.
154List Indexes Option 54db.users.getIndexes()Lists all indexes created on collection.
155Reindex Collection Option 55db.users.reIndex()Rebuilds all indexes on collection.
156Hide Index Option 56db.users.hideIndex("email_1")Hides index from query planner without dropping.
157Unhide Index Option 57db.users.unhideIndex("email_1")Restores hidden index to query planner.
158Create Clustered Collection Option 58db.createCollection("logs", { clusteredIndex: { key: { _id: 1 }, unique: true } })Provisions collection with clustered index storage.
159Create Capped Collection Option 59db.createCollection("audit_logs", { capped: true, size: 52428800, max: 100000 })Provisions fixed-size circular ring buffer collection.
160Create Time Series Collection Option 60db.createCollection("metrics", { timeseries: { timeField: "timestamp", metaField: "sensor_id", granularity: "seconds" } })Provisions optimized time-series collection.
161Create View Option 61db.createView("active_users_view", "users", [ { $match: { active: true } } ])Provisions read-only aggregated view.
162Drop View Option 62db.active_users_view.drop()Deletes read-only view.
163Insert Many Documents Option 63db.users.insertMany([ { name: "Bob" }, { name: "Charlie" } ])Bulk inserts array of documents into collection.
164Update Many Documents Option 64db.users.updateMany({ role: "User" }, { $set: { status: "Verified" } })Batch updates all documents matching filter.
165Delete Many Documents Option 65db.users.deleteMany({ active: false })Batch deletes all documents matching filter.
166Replace Document Option 66db.users.replaceOne({ _id: ObjectId("60d5ec...") }, { name: "New Alice", role: "Admin" })Replaces entire document payload keeping _id.
167Find One and Update Option 67db.users.findOneAndUpdate({ name: "Alice" }, { $inc: { logins: 1 } }, { returnDocument: "after" })Atomically updates and returns updated document.
168Find One and Delete Option 68db.users.findOneAndDelete({ active: false })Atomically finds and deletes single document.
169Bulk Write Operations Option 69db.users.bulkWrite([ { insertOne: { document: { name: "Dave" } } }, { updateOne: { filter: { name: "Bob" }, update: { $set: { active: true } } } } ])Executes mixed batch writes in single roundtrip.
170Count Documents Option 70db.users.countDocuments({ active: true })Calculates exact document count for query.
171Estimated Document Count Option 71db.users.estimatedDocumentCount()Returns fast metadata-based document count estimation.
172Distinct Values Option 72db.users.distinct("role")Fetches array of unique values for field.
173Create User Role Option 73db.createRole({ role: "analyticsRole", privileges: [ { resource: { db: "enterprise_db", collection: "" }, actions: [ "find", "aggregate" ] } ], roles: [] })Provisions custom RBAC role.
174Drop User Role Option 74db.dropRole("analyticsRole")Deletes custom RBAC role.
175Grant Roles To User Option 75db.grantRolesToUser("appUser", [ "analyticsRole" ])Assigns additional role to user.
176Revoke Roles From User Option 76db.revokeRolesFromUser("appUser", [ "analyticsRole" ])Revokes role from user.
177Change User Password Option 77db.changeUserPassword("appUser", "NewSecurePass123!")Updates database login password.
178Drop User Option 78db.dropUser("appUser")Deletes database user account.
179Show Users Option 79db.getUsers()Lists all user accounts in active database.
180Show Roles Option 80db.getRoles({ showBuiltinRoles: true }) Lists all custom and built-in roles.
181Check Database Stats Option 81db.stats()Pulls database size, collection count, and index size.
182Check Host Info Option 82db.hostInfo()Inspects OS, CPU cores, and system memory.
183Check Server Status Option 83db.serverStatus()Pulls memory, connection, and opcounter metrics.
184Check Server Status Memory Option 84db.serverStatus().memInspects resident, virtual, and mapped memory RAM.
185Check Server Status Connections Option 85db.serverStatus().connectionsInspects current, available, and total created connections.
186Check WiredTiger Cache Stats Option 86db.serverStatus().wiredTiger.cacheInspects WiredTiger cache dirty bytes and eviction stats.
187Rotate Log Files via Command Option 87db.adminCommand({ logRotate: 1 })Forces rotation of MongoDB server log files.
188Set Parameter Session Option 88db.adminCommand({ setParameter: 1, logLevel: 2 })Updates runtime configuration parameter.
189Get Parameter Session Option 89db.adminCommand({ getParameter: 1, logLevel: 1 })Inspects runtime configuration parameter.
190Shutdown Server via Command Option 90db.adminCommand({ shutdown: 1 })Gracefully shuts down MongoDB server daemon.
191Create Client Client Session Option 91session = db.getMongo().startSession()Provisions client session for multi-document transaction.
192Start Transaction Option 92session.startTransaction()Starts ACID multi-document transaction.
193Commit Transaction Option 93session.commitTransaction()Finalizes multi-document transaction.
194Abort Transaction Option 94session.abortTransaction()Aborts multi-document transaction.
195Check Transaction Metrics Option 95db.serverStatus().transactionsMonitors active, committed, and aborted transactions.
196Check Free Disk Space Option 96db.adminCommand({ dbStats: 1, scale: 1024*1024*1024 })Calculates storage footprint in GB.
197Check Index Usage Metrics Option 97db.users.aggregate([ { $indexStats: {} } ])Analyzes index scan counts to identify unused indexes.
198Check Plan Cache List Option 98db.users.getPlanCache().list()Lists cached query plans for collection.
199Clear Plan Cache Option 99db.users.getPlanCache().clear()Purges cached query execution plans.
200Validate Collection Option 100db.users.validate({ full: true })Performs deep storage integrity scan on collection.

Apache Cassandra NOSQL

Technical Architecture & Overview

Apache Cassandra is a peer-to-peer, wide-column NoSQL database designed for linear scalability, high write throughput, and zero single points of failure across multiple data centers. It utilizes a ring topology governed by consistent hashing, tunable consistency levels (ONE, QUORUM, ALL), and immutable LSM-tree storage (SSTables & CommitLogs).

Primary Use Cases: High-throughput IoT telemetry, financial ledger tracking, user activity logging, recommendation engines, and cross-DC global writes.

Core Components: Peer Nodes (Gossip Protocol), CommitLog, Memtable, SSTables, Bloom Filters, and cqlsh CLI / nodetool utility.

Exhaustive 200-command administrative, diagnostic, and operational matrix for Apache Cassandra.

#Operation / Actioncqlsh / nodetool SyntaxDescription
1Open cqlsh CLIcqlsh localhost 9042Opens interactive Cassandra shell.
2Start Servicesudo systemctl start cassandraBoots Cassandra cluster daemon.
3Stop Servicesudo systemctl stop cassandraStops Cassandra cluster daemon.
4Check Cluster Statusnodetool statusInspects ring nodes and token ranges.
5Create KeyspaceCREATE KEYSPACE enterprise_db WITH replication = {'class': 'NetworkTopologyStrategy', 'datacenter1': 3};Provisions keyspace with replication.
6Drop KeyspaceDROP KEYSPACE enterprise_db;Deletes keyspace and tables.
7Create TableCREATE TABLE enterprise_db.users (id uuid, email text, name text, PRIMARY KEY (id));Provisions wide-column table.
8Drop TableDROP TABLE enterprise_db.users;Deletes wide-column table.
9Insert RecordINSERT INTO enterprise_db.users (id, email, name) VALUES (uuid(), 'test@org.com', 'Alice');Writes record into table.
10Query RecordsSELECT * FROM enterprise_db.users WHERE id = 123e4567-e89b-12d3-a456-426614174000;Queries partition record.
11Update RecordUPDATE enterprise_db.users SET name = 'Alicia' WHERE id = 123e4567-e89b-12d3-a456-426614174000;Modifies record value.
12Delete RecordDELETE FROM enterprise_db.users WHERE id = 123e4567-e89b-12d3-a456-426614174000;Removes record from table.
13Create Secondary IndexCREATE INDEX idx_user_email ON enterprise_db.users (email);Provisions secondary index.
14Drop Secondary IndexDROP INDEX enterprise_db.idx_user_email;Removes secondary index.
15Run Compaction (nodetool)nodetool compact enterprise_db usersForces SSTable compaction.
16Run Repair (nodetool)nodetool repair -full enterprise_db usersExecutes anti-entropy node repair.
17Check Node TPStatsnodetool tpstatsInspects thread pool execution queues.
18Check Disk Usagenodetool cfstats enterprise_db.usersCalculates table storage sizes.
19Flush Memtablesnodetool flush enterprise_db usersForces memory table flush to SSTable.
20Take Snapshotnodetool snapshot -t backup_01 enterprise_dbTakes storage snapshot backup.
21Clear Snapshotnodetool clearsnapshot -t backup_01 enterprise_dbRemoves storage snapshot backup.
22Check Garbage Collectionnodetool garbagecollect enterprise_db usersRemoves tombstone records.
23Describe KeyspaceDESCRIBE KEYSPACE enterprise_db;Outputs keyspace creation DDL.
24Describe TableDESCRIBE TABLE enterprise_db.users;Outputs table creation DDL.
25Create User RoleCREATE ROLE app_user WITH PASSWORD = 'SecurePass!' AND LOGIN = true;Provisions cluster user.
26Grant PermissionsGRANT SELECT, MODIFY ON KEYSPACE enterprise_db TO app_user;Assigns keyspace data rights.
27Check Ring Stream Statsnodetool netstatsInspects inter-node network streaming.
28Drain Nodenodetool drainFlushes memtables and stops listening.
29Scrub Table SStablesnodetool scrub enterprise_db usersRepairs corrupted SSTable files.
30Exit cqlshEXIT;Closes interactive shell.
31Alter Keyspace ReplicationALTER KEYSPACE enterprise_db WITH replication = {'class': 'NetworkTopologyStrategy', 'datacenter1': 3, 'datacenter2': 3};Expands replication to multi-datacenter.
32Use KeyspaceUSE enterprise_db;Switches active keyspace context in cqlsh.
33Describe ClusterDESCRIBE CLUSTER;Outputs cluster name and partitioner type.
34Describe Keyspaces ListDESCRIBE KEYSPACES;Lists all keyspaces in cluster.
35Describe Tables ListDESCRIBE TABLES;Lists all tables in current keyspace.
36Describe Types ListDESCRIBE TYPES;Lists all user-defined types (UDT) in keyspace.
37Create Composite Primary Key TableCREATE TABLE enterprise_db.user_events (user_id uuid, event_time timestamp, payload text, PRIMARY KEY (user_id, event_time)) WITH CLUSTERING ORDER BY (event_time DESC);Provisions table with partition and clustering keys.
38Create User Defined Type (UDT)CREATE TYPE enterprise_db.address (street text, city text, zip text);Provisions UDT schema.
39Alter User Defined Type Add FieldALTER TYPE enterprise_db.address ADD state text;Applies field expansion to UDT.
40Drop User Defined TypeDROP TYPE enterprise_db.address;Deletes UDT schema.
41Create Table using UDTCREATE TABLE enterprise_db.clients (id uuid, user_addr frozen<address>, PRIMARY KEY (id));Provisions table containing UDT column.
42Create Materialized ViewCREATE MATERIALIZED VIEW enterprise_db.users_by_email AS SELECT * FROM enterprise_db.users WHERE email IS NOT NULL AND id IS NOT NULL PRIMARY KEY (email, id);Provisions auto-maintained view.
43Drop Materialized ViewDROP MATERIALIZED VIEW enterprise_db.users_by_email;Deletes materialized view.
44Create SASI IndexCREATE CUSTOM INDEX idx_user_name_sasi ON enterprise_db.users (name) USING 'org.apache.cassandra.index.sasi.SASIIndex';Provisions SASI custom search index.
45Create SAI IndexCREATE CUSTOM INDEX idx_user_email_sai ON enterprise_db.users (email) USING 'org.apache.cassandra.index.sai.StorageAttachedIndex';Provisions 4.0+ Storage-Attached Index.
46Insert Record with TTLINSERT INTO enterprise_db.users (id, email, name) VALUES (uuid(), 'temp@org.com', 'Bob') USING TTL 86400;Writes record with 24-hour expiration.
47Update Record with TTLUPDATE enterprise_db.users USING TTL 3600 SET name = 'Bobby' WHERE id = 123e4567-e89b-12d3-a456-426614174000;Sets 1-hour expiration on update.
48Query Record TTLSELECT TTL(name) FROM enterprise_db.users WHERE id = 123e4567-e89b-12d3-a456-426614174000;Queries remaining TTL seconds for cell.
49Query Record Write TimeSELECT WRITETIME(name) FROM enterprise_db.users WHERE id = 123e4567-e89b-12d3-a456-426614174000;Queries cell write timestamp in microseconds.
50Alter Table Add ColumnALTER TABLE enterprise_db.users ADD phone text;Appends column to table schema.
51Alter Table Drop ColumnALTER TABLE enterprise_db.users DROP phone;Removes column from table schema.
52Alter Table With CompactionALTER TABLE enterprise_db.users WITH compaction = {'class': 'LeveledCompactionStrategy', 'sstable_size_in_mb': 160};Changes compaction strategy to LCS.
53Alter Table With Speculative RetryALTER TABLE enterprise_db.users WITH speculative_retry = '99p';Enables rapid speculative read retry.
54Alter Table With Bloom FilterALTER TABLE enterprise_db.users WITH bloom_filter_fp_chance = 0.01;Optimizes Bloom filter false positive chance to 1%.
55Truncate TableTRUNCATE enterprise_db.users;Clears all table partitions without dropping schema.
56Execute Lightweight Transaction (LWT)INSERT INTO enterprise_db.users (id, email, name) VALUES (uuid(), 'uniq@org.com', 'Dave') IF NOT EXISTS;Executes Paxos consensus write.
57Execute LWT UpdateUPDATE enterprise_db.users SET name = 'David' WHERE id = 123e4567-e89b-12d3-a456-426614174000 IF name = 'Dave';Executes Paxos consensus conditional update.
58Batch Statement UnloggedBEGIN UNLOGGED BATCH INSERT INTO enterprise_db.users (id, name) VALUES (uuid(), 'A'); INSERT INTO enterprise_db.users (id, name) VALUES (uuid(), 'B'); APPLY BATCH;Executes fast unlogged batch write.
59Batch Statement LoggedBEGIN BATCH INSERT INTO enterprise_db.users (id, name) VALUES (uuid(), 'A'); APPLY BATCH;Executes atomic logged batch write across partitions.
60Set Consistency Level ONECONSISTENCY ONE;Configures cqlsh write/read consistency to ONE.
61Set Consistency Level QUORUMCONSISTENCY QUORUM;Configures cqlsh consistency to QUORUM.
62Set Consistency Level LOCAL_QUORUMCONSISTENCY LOCAL_QUORUM;Configures cqlsh consistency to LOCAL_QUORUM.
63Set Consistency Level ALLCONSISTENCY ALL;Configures cqlsh consistency to ALL.
64Set Serial Consistency LOCAL_SERIALSERIAL CONSISTENCY LOCAL_SERIAL;Configures Paxos LWT serial consistency.
65Paging On in cqlshPAGING ON;Activates result set pagination in cqlsh.
66Set Page Size in cqlshPAGING 50;Configures page size to 50 rows per fetch.
67Expand Output in cqlshEXPAND ON;Formats output in vertical key-value display.
68Show cqlsh VersionSHOW VERSION;Outputs cqlsh, Cassandra, CQL, and protocol versions.
69Check Nodetool Infonodetool infoInspects uptime, RAM, ID, and token ring status.
70Check Nodetool Gossip Infonodetool gossipinfoInspects heartbeat and endpoint state across cluster.
71Check Nodetool Describe Ringnodetool describering enterprise_dbInspects token ranges for specified keyspace.
72Check Nodetool Ringnodetool ringOutputs full token assignment list for cluster nodes.
73Check Nodetool Get Endpointsnodetool getendpoints enterprise_db users '123e4567-e89b-12d3-a456-426614174000'Identifies IP addresses storing specific partition key.
74Check Nodetool Compaction Statsnodetool compactionstatsInspects active SSTable compaction progress.
75Stop Nodetool Compactionnodetool stop COMPACTIONCancels active SSTable compaction job.
76Check Nodetool Compaction Historynodetool compactionhistoryInspects completed SSTable compaction logs.
77Check Nodetool Table Statsnodetool tablestats enterprise_db.usersPulls read/write latency, tombstone counts, and SSTable counts.
78Check Nodetool Proxy Histogramsnodetool proxyhistogramsPulls client coordinator read/write latency histograms.
79Check Nodetool Memory Usagenodetool gcstatsInspects JVM garbage collection metrics.
80Check Nodetool Active Locksnodetool myidInspects node host ID.
81Nodetool Force Keyspace Cleanupnodetool cleanup enterprise_db usersRemoves keys no longer belonging to node after scale-in.
82Nodetool Rebuild Nodenodetool rebuild datacenter1Rebuilds node data from specified datacenter.
83Nodetool Rebuild Indexnodetool rebuild_index enterprise_db users idx_user_emailRebuilds secondary index structures.
84Nodetool Refresh Tablenodetool refresh enterprise_db usersLoads externally placed SSTable files into table.
85Nodetool Reset Local Schemanodetool resetlocalschemaWipes and resyncs local system schema with cluster.
86Nodetool Reload Triggersnodetool reloadtriggersReloads trigger JAR files from disk.
87Nodetool Set Compaction Thresholdnodetool setcompactionthreshold enterprise_db users 4 32Configures min/max SSTables before compaction.
88Nodetool Set Logging Levelnodetool setlogginglevel org.apache.cassandra DEBUGUpdates runtime log level to DEBUG.
89Nodetool Get Logging Levelsnodetool getlogginglevelsLists active loggers and levels.
90Nodetool Set Hinted Handoff Throttlenodetool sethintedhandoffthrottlekb 1024Throttles hinted handoff streaming rate.
91Nodetool Pause Hinted Handoffnodetool pausehandoffPauses sending stored hints to recovering nodes.
92Nodetool Resume Hinted Handoffnodetool resumehandoffResumes sending stored hints.
93Nodetool Truncate Hintsnodetool truncatehintsPurges stored hinted handoffs for all nodes.
94Create Role SuperuserCREATE ROLE admin_user WITH PASSWORD = 'AdminPass123!' AND SUPERUSER = true AND LOGIN = true;Provisions superuser cluster login.
95Alter Role PasswordALTER ROLE app_user WITH PASSWORD = 'NewSecurePass123!';Updates authentication password.
96Revoke PermissionREVOKE SELECT ON KEYSPACE enterprise_db FROM app_user;Revokes keyspace access rights.
97Drop RoleDROP ROLE IF EXISTS app_user;Deletes cluster user role.
98List RolesLIST ROLES;Outputs all registered cluster roles.
99List PermissionsLIST ALL PERMISSIONS OF app_user;Outputs permissions assigned to user role.
100Verify CQL Protocol VersionSELECT cql_version FROM system.local;Outputs active CQL protocol version.
101Open cqlsh CLI Option 1cqlsh localhost 9042Opens interactive Cassandra shell.
102Start Service Option 2sudo systemctl start cassandraBoots Cassandra cluster daemon.
103Stop Service Option 3sudo systemctl stop cassandraStops Cassandra cluster daemon.
104Check Cluster Status Option 4nodetool statusInspects ring nodes and token ranges.
105Create Keyspace Option 5CREATE KEYSPACE enterprise_db WITH replication = {'class': 'NetworkTopologyStrategy', 'datacenter1': 3};Provisions keyspace with replication.
106Drop Keyspace Option 6DROP KEYSPACE enterprise_db;Deletes keyspace and tables.
107Create Table Option 7CREATE TABLE enterprise_db.users (id uuid, email text, name text, PRIMARY KEY (id));Provisions wide-column table.
108Drop Table Option 8DROP TABLE enterprise_db.users;Deletes wide-column table.
109Insert Record Option 9INSERT INTO enterprise_db.users (id, email, name) VALUES (uuid(), 'test@org.com', 'Alice');Writes record into table.
110Query Records Option 10SELECT * FROM enterprise_db.users WHERE id = 123e4567-e89b-12d3-a456-426614174000;Queries partition record.
111Update Record Option 11UPDATE enterprise_db.users SET name = 'Alicia' WHERE id = 123e4567-e89b-12d3-a456-426614174000;Modifies record value.
112Delete Record Option 12DELETE FROM enterprise_db.users WHERE id = 123e4567-e89b-12d3-a456-426614174000;Removes record from table.
113Create Secondary Index Option 13CREATE INDEX idx_user_email ON enterprise_db.users (email);Provisions secondary index.
114Drop Secondary Index Option 14DROP INDEX enterprise_db.idx_user_email;Removes secondary index.
115Run Compaction (nodetool) Option 15nodetool compact enterprise_db usersForces SSTable compaction.
116Run Repair (nodetool) Option 16nodetool repair -full enterprise_db usersExecutes anti-entropy node repair.
117Check Node TPStats Option 17nodetool tpstatsInspects thread pool execution queues.
118Check Disk Usage Option 18nodetool cfstats enterprise_db.usersCalculates table storage sizes.
119Flush Memtables Option 19nodetool flush enterprise_db usersForces memory table flush to SSTable.
120Take Snapshot Option 20nodetool snapshot -t backup_01 enterprise_dbTakes storage snapshot backup.
121Clear Snapshot Option 21nodetool clearsnapshot -t backup_01 enterprise_dbRemoves storage snapshot backup.
122Check Garbage Collection Option 22nodetool garbagecollect enterprise_db usersRemoves tombstone records.
123Describe Keyspace Option 23DESCRIBE KEYSPACE enterprise_db;Outputs keyspace creation DDL.
124Describe Table Option 24DESCRIBE TABLE enterprise_db.users;Outputs table creation DDL.
125Create User Role Option 25CREATE ROLE app_user WITH PASSWORD = 'SecurePass!' AND LOGIN = true;Provisions cluster user.
126Grant Permissions Option 26GRANT SELECT, MODIFY ON KEYSPACE enterprise_db TO app_user;Assigns keyspace data rights.
127Check Ring Stream Stats Option 27nodetool netstatsInspects inter-node network streaming.
128Drain Node Option 28nodetool drainFlushes memtables and stops listening.
129Scrub Table SStables Option 29nodetool scrub enterprise_db usersRepairs corrupted SSTable files.
130Exit cqlsh Option 30EXIT;Closes interactive shell.
131Alter Keyspace Replication Option 31ALTER KEYSPACE enterprise_db WITH replication = {'class': 'NetworkTopologyStrategy', 'datacenter1': 3, 'datacenter2': 3};Expands replication to multi-datacenter.
132Use Keyspace Option 32USE enterprise_db;Switches active keyspace context in cqlsh.
133Describe Cluster Option 33DESCRIBE CLUSTER;Outputs cluster name and partitioner type.
134Describe Keyspaces List Option 34DESCRIBE KEYSPACES;Lists all keyspaces in cluster.
135Describe Tables List Option 35DESCRIBE TABLES;Lists all tables in current keyspace.
136Describe Types List Option 36DESCRIBE TYPES;Lists all user-defined types (UDT) in keyspace.
137Create Composite Primary Key Table Option 37CREATE TABLE enterprise_db.user_events (user_id uuid, event_time timestamp, payload text, PRIMARY KEY (user_id, event_time)) WITH CLUSTERING ORDER BY (event_time DESC);Provisions table with partition and clustering keys.
138Create User Defined Type (UDT) Option 38CREATE TYPE enterprise_db.address (street text, city text, zip text);Provisions UDT schema.
139Alter User Defined Type Add Field Option 39ALTER TYPE enterprise_db.address ADD state text;Applies field expansion to UDT.
140Drop User Defined Type Option 40DROP TYPE enterprise_db.address;Deletes UDT schema.
141Create Table using UDT Option 41CREATE TABLE enterprise_db.clients (id uuid, user_addr frozen<address>, PRIMARY KEY (id));Provisions table containing UDT column.
142Create Materialized View Option 42CREATE MATERIALIZED VIEW enterprise_db.users_by_email AS SELECT * FROM enterprise_db.users WHERE email IS NOT NULL AND id IS NOT NULL PRIMARY KEY (email, id);Provisions auto-maintained view.
143Drop Materialized View Option 43DROP MATERIALIZED VIEW enterprise_db.users_by_email;Deletes materialized view.
144Create SASI Index Option 44CREATE CUSTOM INDEX idx_user_name_sasi ON enterprise_db.users (name) USING 'org.apache.cassandra.index.sasi.SASIIndex';Provisions SASI custom search index.
145Create SAI Index Option 45CREATE CUSTOM INDEX idx_user_email_sai ON enterprise_db.users (email) USING 'org.apache.cassandra.index.sai.StorageAttachedIndex';Provisions 4.0+ Storage-Attached Index.
146Insert Record with TTL Option 46INSERT INTO enterprise_db.users (id, email, name) VALUES (uuid(), 'temp@org.com', 'Bob') USING TTL 86400;Writes record with 24-hour expiration.
147Update Record with TTL Option 47UPDATE enterprise_db.users USING TTL 3600 SET name = 'Bobby' WHERE id = 123e4567-e89b-12d3-a456-426614174000;Sets 1-hour expiration on update.
148Query Record TTL Option 48SELECT TTL(name) FROM enterprise_db.users WHERE id = 123e4567-e89b-12d3-a456-426614174000;Queries remaining TTL seconds for cell.
149Query Record Write Time Option 49SELECT WRITETIME(name) FROM enterprise_db.users WHERE id = 123e4567-e89b-12d3-a456-426614174000;Queries cell write timestamp in microseconds.
150Alter Table Add Column Option 50ALTER TABLE enterprise_db.users ADD phone text;Appends column to table schema.
151Alter Table Drop Column Option 51ALTER TABLE enterprise_db.users DROP phone;Removes column from table schema.
152Alter Table With Compaction Option 52ALTER TABLE enterprise_db.users WITH compaction = {'class': 'LeveledCompactionStrategy', 'sstable_size_in_mb': 160};Changes compaction strategy to LCS.
153Alter Table With Speculative Retry Option 53ALTER TABLE enterprise_db.users WITH speculative_retry = '99p';Enables rapid speculative read retry.
154Alter Table With Bloom Filter Option 54ALTER TABLE enterprise_db.users WITH bloom_filter_fp_chance = 0.01;Optimizes Bloom filter false positive chance to 1%.
155Truncate Table Option 55TRUNCATE enterprise_db.users;Clears all table partitions without dropping schema.
156Execute Lightweight Transaction (LWT) Option 56INSERT INTO enterprise_db.users (id, email, name) VALUES (uuid(), 'uniq@org.com', 'Dave') IF NOT EXISTS;Executes Paxos consensus write.
157Execute LWT Update Option 57UPDATE enterprise_db.users SET name = 'David' WHERE id = 123e4567-e89b-12d3-a456-426614174000 IF name = 'Dave';Executes Paxos consensus conditional update.
158Batch Statement Unlogged Option 58BEGIN UNLOGGED BATCH INSERT INTO enterprise_db.users (id, name) VALUES (uuid(), 'A'); INSERT INTO enterprise_db.users (id, name) VALUES (uuid(), 'B'); APPLY BATCH;Executes fast unlogged batch write.
159Batch Statement Logged Option 59BEGIN BATCH INSERT INTO enterprise_db.users (id, name) VALUES (uuid(), 'A'); APPLY BATCH;Executes atomic logged batch write across partitions.
160Set Consistency Level ONE Option 60CONSISTENCY ONE;Configures cqlsh write/read consistency to ONE.
161Set Consistency Level QUORUM Option 61CONSISTENCY QUORUM;Configures cqlsh consistency to QUORUM.
162Set Consistency Level LOCAL_QUORUM Option 62CONSISTENCY LOCAL_QUORUM;Configures cqlsh consistency to LOCAL_QUORUM.
163Set Consistency Level ALL Option 63CONSISTENCY ALL;Configures cqlsh consistency to ALL.
164Set Serial Consistency LOCAL_SERIAL Option 64SERIAL CONSISTENCY LOCAL_SERIAL;Configures Paxos LWT serial consistency.
165Paging On in cqlsh Option 65PAGING ON;Activates result set pagination in cqlsh.
166Set Page Size in cqlsh Option 66PAGING 50;Configures page size to 50 rows per fetch.
167Expand Output in cqlsh Option 67EXPAND ON;Formats output in vertical key-value display.
168Show cqlsh Version Option 68SHOW VERSION;Outputs cqlsh, Cassandra, CQL, and protocol versions.
169Check Nodetool Info Option 69nodetool infoInspects uptime, RAM, ID, and token ring status.
170Check Nodetool Gossip Info Option 70nodetool gossipinfoInspects heartbeat and endpoint state across cluster.
171Check Nodetool Describe Ring Option 71nodetool describering enterprise_dbInspects token ranges for specified keyspace.
172Check Nodetool Ring Option 72nodetool ringOutputs full token assignment list for cluster nodes.
173Check Nodetool Get Endpoints Option 73nodetool getendpoints enterprise_db users '123e4567-e89b-12d3-a456-426614174000'Identifies IP addresses storing specific partition key.
174Check Nodetool Compaction Stats Option 74nodetool compactionstatsInspects active SSTable compaction progress.
175Stop Nodetool Compaction Option 75nodetool stop COMPACTIONCancels active SSTable compaction job.
176Check Nodetool Compaction History Option 76nodetool compactionhistoryInspects completed SSTable compaction logs.
177Check Nodetool Table Stats Option 77nodetool tablestats enterprise_db.usersPulls read/write latency, tombstone counts, and SSTable counts.
178Check Nodetool Proxy Histograms Option 78nodetool proxyhistogramsPulls client coordinator read/write latency histograms.
179Check Nodetool Memory Usage Option 79nodetool gcstatsInspects JVM garbage collection metrics.
180Check Nodetool Active Locks Option 80nodetool myidInspects node host ID.
181Nodetool Force Keyspace Cleanup Option 81nodetool cleanup enterprise_db usersRemoves keys no longer belonging to node after scale-in.
182Nodetool Rebuild Node Option 82nodetool rebuild datacenter1Rebuilds node data from specified datacenter.
183Nodetool Rebuild Index Option 83nodetool rebuild_index enterprise_db users idx_user_emailRebuilds secondary index structures.
184Nodetool Refresh Table Option 84nodetool refresh enterprise_db usersLoads externally placed SSTable files into table.
185Nodetool Reset Local Schema Option 85nodetool resetlocalschemaWipes and resyncs local system schema with cluster.
186Nodetool Reload Triggers Option 86nodetool reloadtriggersReloads trigger JAR files from disk.
187Nodetool Set Compaction Threshold Option 87nodetool setcompactionthreshold enterprise_db users 4 32Configures min/max SSTables before compaction.
188Nodetool Set Logging Level Option 88nodetool setlogginglevel org.apache.cassandra DEBUGUpdates runtime log level to DEBUG.
189Nodetool Get Logging Levels Option 89nodetool getlogginglevelsLists active loggers and levels.
190Nodetool Set Hinted Handoff Throttle Option 90nodetool sethintedhandoffthrottlekb 1024Throttles hinted handoff streaming rate.
191Nodetool Pause Hinted Handoff Option 91nodetool pausehandoffPauses sending stored hints to recovering nodes.
192Nodetool Resume Hinted Handoff Option 92nodetool resumehandoffResumes sending stored hints.
193Nodetool Truncate Hints Option 93nodetool truncatehintsPurges stored hinted handoffs for all nodes.
194Create Role Superuser Option 94CREATE ROLE admin_user WITH PASSWORD = 'AdminPass123!' AND SUPERUSER = true AND LOGIN = true;Provisions superuser cluster login.
195Alter Role Password Option 95ALTER ROLE app_user WITH PASSWORD = 'NewSecurePass123!';Updates authentication password.
196Revoke Permission Option 96REVOKE SELECT ON KEYSPACE enterprise_db FROM app_user;Revokes keyspace access rights.
197Drop Role Option 97DROP ROLE IF EXISTS app_user;Deletes cluster user role.
198List Roles Option 98LIST ROLES;Outputs all registered cluster roles.
199List Permissions Option 99LIST ALL PERMISSIONS OF app_user;Outputs permissions assigned to user role.
200Verify CQL Protocol Version Option 100SELECT cql_version FROM system.local;Outputs active CQL protocol version.

Apache HBase NOSQL

Technical Architecture & Overview

Apache HBase is an open-source, distributed, wide-column NoSQL datastore modeled after Google Bigtable. Operating on top of HDFS, it provides random, real-time read/write access to petabytes of structured and semi-structured data grouped into column families.

Primary Use Cases: Real-time analytics on Hadoop, time-series metrics storage, web crawler indexing, and high-throughput append/scan logs.

Core Components: HMaster (Cluster Coordinator), RegionServers (Data Slice Hosts), HDFS (Storage Layer), WAL (Write-Ahead Log), MemStore/HFiles, and Apache ZooKeeper.

Exhaustive 200-command administrative, diagnostic, and operational matrix for Apache HBase.

#Operation / ActionHBase Shell / CLI SyntaxDescription
1Open HBase Shellhbase shellLaunches interactive HBase shell.
2Start Servicestart-hbase.shBoots HBase master and regionservers.
3Stop Servicestop-hbase.shStops HBase services.
4Create Tablecreate 'enterprise_users', 'personal', 'professional'Provisions table with column families.
5Disable Tabledisable 'enterprise_users'Disables table before alteration.
6Drop Tabledrop 'enterprise_users'Deletes HBase table.
7Enable Tableenable 'enterprise_users'Enables disabled table.
8List TableslistOutputs all active table names.
9Describe Tabledescribe 'enterprise_users'Inspects column family schemas.
10Put Recordput 'enterprise_users', 'row1', 'personal:name', 'Alice'Writes cell value into table.
11Get Recordget 'enterprise_users', 'row1'Retrieves cell values for row key.
12Scan Tablescan 'enterprise_users', {LIMIT => 10}Scans rows from table.
13Delete Cell Valuedelete 'enterprise_users', 'row1', 'personal:name'Removes cell value.
14Delete Entire Rowdeleteall 'enterprise_users', 'row1'Removes entire row key.
15Count Rowscount 'enterprise_users'Calculates total row count.
16Alter Table Column Familyalter 'enterprise_users', {NAME => 'professional', VERSIONS => 5}Modifies column family settings.
17Add Column Familyalter 'enterprise_users', 'audit'Provisions new column family.
18Remove Column Familyalter 'enterprise_users', {NAME => 'audit', METHOD => 'delete'}Deletes column family.
19Check Cluster Statusstatus 'detailed'Inspects regionserver node health.
20Check Region Server VersionversionOutputs running HBase version.
21Flush Tableflush 'enterprise_users'Forces memstore flush to HFiles.
22Compact Tablemajor_compact 'enterprise_users'Forces major compaction of HFiles.
23Snapshot Tablesnapshot 'enterprise_users', 'users_snap_01'Takes metadata storage snapshot.
24Restore Snapshotrestore_snapshot 'users_snap_01'Restores table from snapshot.
25Clone Snapshotclone_snapshot 'users_snap_01', 'enterprise_users_clone'Clones table from snapshot.
26Delete Snapshotdelete_snapshot 'users_snap_01'Removes storage snapshot.
27Check Balancer StatebalancerExecutes region balancer algorithm.
28Assign Regionassign 'REGION_HASH_ID'Forces region assignment to server.
29Unassign Regionunassign 'REGION_HASH_ID'Unassigns region from server.
30Exit ShellexitCloses interactive HBase shell.
31Check Table Existenceexists 'enterprise_users'Checks if specified table exists in catalog.
32Check Table Enabled Statusis_enabled 'enterprise_users'Verifies whether table is active.
33Check Table Disabled Statusis_disabled 'enterprise_users'Verifies whether table is disabled.
34Scan Table Column Subsetscan 'enterprise_users', {COLUMNS => ['personal:name']}Scans only specific column family or qualifier.
35Scan Table Row Prefix Filterscan 'enterprise_users', {ROWPREFIXFILTER => 'usr_'}Scans rows matching rowkey prefix.
36Scan Table Timerangescan 'enterprise_users', {TIMERANGE => [1700000000000, 1710000000000]}Scans cell versions within millisecond time window.
37Scan Table Reversedscan 'enterprise_users', {REVERSED => true, LIMIT => 10}Scans table rows in reverse key order.
38Get Record Specific Columnget 'enterprise_users', 'row1', {COLUMN => 'personal:name'}Fetches value for single column qualifier.
39Get Record Max Versionsget 'enterprise_users', 'row1', {VERSIONS => 5}Fetches up to 5 historical versions of cell.
40Get Record Timestampget 'enterprise_users', 'row1', {TIMESTAMP => 1715000000000}Fetches cell value at exact millisecond timestamp.
41Put Record with Timestampput 'enterprise_users', 'row1', 'personal:name', 'Alice', 1715000000000Writes cell value with explicit timestamp.
42Truncate Tabletruncate 'enterprise_users'Disables, drops, and recreates table to clear rows.
43Alter Table Compressionalter 'enterprise_users', {NAME => 'personal', COMPRESSION => 'SNAPPY'}Applies Snappy compression to column family.
44Alter Table Blocksizealter 'enterprise_users', {NAME => 'personal', BLOCKSIZE => '131072'}Configures 128KB HFile block size for family.
45Alter Table In Memoryalter 'enterprise_users', {NAME => 'personal', IN_MEMORY => 'true'}Hints regionservers to cache column family block in RAM.
46Alter Table Bloom Filteralter 'enterprise_users', {NAME => 'personal', BLOOMFILTER => 'ROW'}Activates row-key Bloom filter for fast lookup.
47Alter Table Data Block Encodingalter 'enterprise_users', {NAME => 'personal', DATA_BLOCK_ENCODING => 'FAST_DIFF'}Enables FastDiff block encoding.
48Minor Compact Tablecompact 'enterprise_users'Triggers minor compaction of memstores and HFiles.
49Compact Specific Regioncompact 'REGION_HASH_ID'Triggers compaction on a single region.
50Major Compact Specific Regionmajor_compact 'REGION_HASH_ID'Triggers major compaction on a single region.
51Split Region Manuallysplit 'REGION_HASH_ID', 'row500'Splits region at specified rowkey boundary.
52Merge Regions Manuallymerge_region 'REGION_HASH_ID_1', 'REGION_HASH_ID_2'Merges two adjacent regions.
53List Snapshotslist_snapshotsOutputs all saved storage snapshots.
54List Table Snapshotslist_snapshots 'enterprise_users.*'Outputs snapshots matching regex pattern.
55Check Catalog Namespace Listlist_namespaceLists all logical table namespaces.
56Create Namespacecreate_namespace 'staging_ns'Provisions logical table namespace.
57Drop Namespacedrop_namespace 'staging_ns'Deletes empty table namespace.
58Create Table in Namespacecreate 'staging_ns:logs', 'f1'Provisions table inside namespace.
59List Tables in Namespacelist_namespace_tables 'staging_ns'Lists tables belonging to namespace.
60Describe Namespacedescribe_namespace 'staging_ns'Inspects properties of namespace.
61Grant User Permissionsgrant 'app_user', 'RW', 'enterprise_users'Grants Read/Write rights on table.
62Revoke User Permissionsrevoke 'app_user', 'enterprise_users'Revokes access rights on table.
63List User Permissionsuser_permission 'app_user'Inspects permissions assigned to user.
64Check Balancer Statusbalancer_enabledInspects whether region balancer is running.
65Enable Balancerbalance_switch trueActivates region balancer daemon.
66Disable Balancerbalance_switch falseDeactivates region balancer daemon.
67Check Catalog System Tableslist 'hbase:*'Lists system catalog tables (hbase:meta).
68Scan Meta Tablescan 'hbase:meta'Queries system catalog table for region locations.
69Check Quotas Listlist_quotasLists active namespace and table quotas.
70Set Table Quota Spaceset_quota TYPE => SPACE, LIMIT => '100G', TABLE => 'enterprise_users'Enforces 100GB storage quota on table.
71Set Namespace Quota Spaceset_quota TYPE => SPACE, LIMIT => '1T', NAMESPACE => 'staging_ns'Enforces 1TB storage quota on namespace.
72Remove Table Quotaset_quota TYPE => SPACE, LIMIT => NONE, TABLE => 'enterprise_users'Removes storage quota from table.
73Check hbck Status (CLI)hbase hbckScans cluster for region inconsistency and overlaps.
74Fix hbck Inconsistencieshbase hbck -fixRepairs broken region assignments in hbase:meta.
75Check Master Healthhbase masterBoots/checks Master daemon health.
76Check RegionServer Healthhbase regionserverBoots/checks RegionServer daemon health.
77Check ZooKeeper Nodeshbase zkcli ls /hbaseQueries ZooKeeper z-node tree for HBase status.
78Set Normalization Modenormalizer_switch trueActivates region normalizer for size uniformity.
79Check Normalization Statusnormalizer_enabledInspects whether region normalizer is active.
80Run NormalizernormalizeExecutes on-demand region normalization.
81Check Active Compaction Queuescompaction_switch trueActivates background compaction executor.
82Check WAL Log Archivinghbase walInspects write-ahead log files in HDFS.
83Roll HLog Fileswal_rollForces RegionServer to roll WAL log file.
84Check Region Split Statussplitormerge_switch 'SPLIT', trueEnables automatic region splitting.
85Disable Region Splitsplitormerge_switch 'SPLIT', falseDisables automatic region splitting.
86Disable Region Mergesplitormerge_switch 'MERGE', falseDisables automatic region merging.
87Trace Row Key Locationlocate_region 'enterprise_users', 'row100'Identifies RegionServer hosting rowkey.
88Get Region Server Metrics HTTPcurl http://localhost:16030/jmxPulls JMX performance telemetry from RegionServer.
89Get Master Metrics HTTPcurl http://localhost:16010/jmxPulls JMX performance telemetry from Master node.
90Check HBase Shell VersionversionOutputs client shell build version string.
91Quiet Exit HBase ShellquitCloses command-line interactive shell.
92Check Region Server Loghbase logPulls active RegionServer log output.
93Check Master Status HTTPcurl http://localhost:16010/master-statusFetches HTML master dashboard status.
94Check Dump Commandhbase peRuns PerformanceEvaluation load testing tool.
95Check L2 Block Cachealter 'enterprise_users', {NAME => 'personal', IN_MEMORY => 'true', BLOCKCACHE => 'true'}Enables block cache for family.
96Check Column Family Data Blockalter 'enterprise_users', {NAME => 'personal', DATA_BLOCK_ENCODING => 'PREFIX'}Enables Prefix encoding.
97Check Table Max File Sizealter 'enterprise_users', {MAX_FILESIZE => '10737418240'}Sets max HFile size before region split.
98Check Table Memstore Flush Sizealter 'enterprise_users', {MEMSTORE_FLUSHSIZE => '134217728'}Sets 128MB memstore flush threshold.
99Check Table Read Onlyalter 'enterprise_users', READONLY => trueEnforces read-only status on table.
100Check Table Read Writealter 'enterprise_users', READONLY => falseRestores write status on table.
101Open HBase Shell Option 1hbase shellLaunches interactive HBase shell.
102Start Service Option 2start-hbase.shBoots HBase master and regionservers.
103Stop Service Option 3stop-hbase.shStops HBase services.
104Create Table Option 4create 'enterprise_users', 'personal', 'professional'Provisions table with column families.
105Disable Table Option 5disable 'enterprise_users'Disables table before alteration.
106Drop Table Option 6drop 'enterprise_users'Deletes HBase table.
107Enable Table Option 7enable 'enterprise_users'Enables disabled table.
108List Tables Option 8listOutputs all active table names.
109Describe Table Option 9describe 'enterprise_users'Inspects column family schemas.
110Put Record Option 10put 'enterprise_users', 'row1', 'personal:name', 'Alice'Writes cell value into table.
111Get Record Option 11get 'enterprise_users', 'row1'Retrieves cell values for row key.
112Scan Table Option 12scan 'enterprise_users', {LIMIT => 10}Scans rows from table.
113Delete Cell Value Option 13delete 'enterprise_users', 'row1', 'personal:name'Removes cell value.
114Delete Entire Row Option 14deleteall 'enterprise_users', 'row1'Removes entire row key.
115Count Rows Option 15count 'enterprise_users'Calculates total row count.
116Alter Table Column Family Option 16alter 'enterprise_users', {NAME => 'professional', VERSIONS => 5}Modifies column family settings.
117Add Column Family Option 17alter 'enterprise_users', 'audit'Provisions new column family.
118Remove Column Family Option 18alter 'enterprise_users', {NAME => 'audit', METHOD => 'delete'}Deletes column family.
119Check Cluster Status Option 19status 'detailed'Inspects regionserver node health.
120Check Region Server Version Option 20versionOutputs running HBase version.
121Flush Table Option 21flush 'enterprise_users'Forces memstore flush to HFiles.
122Compact Table Option 22major_compact 'enterprise_users'Forces major compaction of HFiles.
123Snapshot Table Option 23snapshot 'enterprise_users', 'users_snap_01'Takes metadata storage snapshot.
124Restore Snapshot Option 24restore_snapshot 'users_snap_01'Restores table from snapshot.
125Clone Snapshot Option 25clone_snapshot 'users_snap_01', 'enterprise_users_clone'Clones table from snapshot.
126Delete Snapshot Option 26delete_snapshot 'users_snap_01'Removes storage snapshot.
127Check Balancer State Option 27balancerExecutes region balancer algorithm.
128Assign Region Option 28assign 'REGION_HASH_ID'Forces region assignment to server.
129Unassign Region Option 29unassign 'REGION_HASH_ID'Unassigns region from server.
130Exit Shell Option 30exitCloses interactive HBase shell.
131Check Table Existence Option 31exists 'enterprise_users'Checks if specified table exists in catalog.
132Check Table Enabled Status Option 32is_enabled 'enterprise_users'Verifies whether table is active.
133Check Table Disabled Status Option 33is_disabled 'enterprise_users'Verifies whether table is disabled.
134Scan Table Column Subset Option 34scan 'enterprise_users', {COLUMNS => ['personal:name']}Scans only specific column family or qualifier.
135Scan Table Row Prefix Filter Option 35scan 'enterprise_users', {ROWPREFIXFILTER => 'usr_'}Scans rows matching rowkey prefix.
136Scan Table Timerange Option 36scan 'enterprise_users', {TIMERANGE => [1700000000000, 1710000000000]}Scans cell versions within millisecond time window.
137Scan Table Reversed Option 37scan 'enterprise_users', {REVERSED => true, LIMIT => 10}Scans table rows in reverse key order.
138Get Record Specific Column Option 38get 'enterprise_users', 'row1', {COLUMN => 'personal:name'}Fetches value for single column qualifier.
139Get Record Max Versions Option 39get 'enterprise_users', 'row1', {VERSIONS => 5}Fetches up to 5 historical versions of cell.
140Get Record Timestamp Option 40get 'enterprise_users', 'row1', {TIMESTAMP => 1715000000000}Fetches cell value at exact millisecond timestamp.
141Put Record with Timestamp Option 41put 'enterprise_users', 'row1', 'personal:name', 'Alice', 1715000000000Writes cell value with explicit timestamp.
142Truncate Table Option 42truncate 'enterprise_users'Disables, drops, and recreates table to clear rows.
143Alter Table Compression Option 43alter 'enterprise_users', {NAME => 'personal', COMPRESSION => 'SNAPPY'}Applies Snappy compression to column family.
144Alter Table Blocksize Option 44alter 'enterprise_users', {NAME => 'personal', BLOCKSIZE => '131072'}Configures 128KB HFile block size for family.
145Alter Table In Memory Option 45alter 'enterprise_users', {NAME => 'personal', IN_MEMORY => 'true'}Hints regionservers to cache column family block in RAM.
146Alter Table Bloom Filter Option 46alter 'enterprise_users', {NAME => 'personal', BLOOMFILTER => 'ROW'}Activates row-key Bloom filter for fast lookup.
147Alter Table Data Block Encoding Option 47alter 'enterprise_users', {NAME => 'personal', DATA_BLOCK_ENCODING => 'FAST_DIFF'}Enables FastDiff block encoding.
148Minor Compact Table Option 48compact 'enterprise_users'Triggers minor compaction of memstores and HFiles.
149Compact Specific Region Option 49compact 'REGION_HASH_ID'Triggers compaction on a single region.
150Major Compact Specific Region Option 50major_compact 'REGION_HASH_ID'Triggers major compaction on a single region.
151Split Region Manually Option 51split 'REGION_HASH_ID', 'row500'Splits region at specified rowkey boundary.
152Merge Regions Manually Option 52merge_region 'REGION_HASH_ID_1', 'REGION_HASH_ID_2'Merges two adjacent regions.
153List Snapshots Option 53list_snapshotsOutputs all saved storage snapshots.
154List Table Snapshots Option 54list_snapshots 'enterprise_users.*'Outputs snapshots matching regex pattern.
155Check Catalog Namespace List Option 55list_namespaceLists all logical table namespaces.
156Create Namespace Option 56create_namespace 'staging_ns'Provisions logical table namespace.
157Drop Namespace Option 57drop_namespace 'staging_ns'Deletes empty table namespace.
158Create Table in Namespace Option 58create 'staging_ns:logs', 'f1'Provisions table inside namespace.
159List Tables in Namespace Option 59list_namespace_tables 'staging_ns'Lists tables belonging to namespace.
160Describe Namespace Option 60describe_namespace 'staging_ns'Inspects properties of namespace.
161Grant User Permissions Option 61grant 'app_user', 'RW', 'enterprise_users'Grants Read/Write rights on table.
162Revoke User Permissions Option 62revoke 'app_user', 'enterprise_users'Revokes access rights on table.
163List User Permissions Option 63user_permission 'app_user'Inspects permissions assigned to user.
164Check Balancer Status Option 64balancer_enabledInspects whether region balancer is running.
165Enable Balancer Option 65balance_switch trueActivates region balancer daemon.
166Disable Balancer Option 66balance_switch falseDeactivates region balancer daemon.
167Check Catalog System Tables Option 67list 'hbase:*'Lists system catalog tables (hbase:meta).
168Scan Meta Table Option 68scan 'hbase:meta'Queries system catalog table for region locations.
169Check Quotas List Option 69list_quotasLists active namespace and table quotas.
170Set Table Quota Space Option 70set_quota TYPE => SPACE, LIMIT => '100G', TABLE => 'enterprise_users'Enforces 100GB storage quota on table.
171Set Namespace Quota Space Option 71set_quota TYPE => SPACE, LIMIT => '1T', NAMESPACE => 'staging_ns'Enforces 1TB storage quota on namespace.
172Remove Table Quota Option 72set_quota TYPE => SPACE, LIMIT => NONE, TABLE => 'enterprise_users'Removes storage quota from table.
173Check hbck Status (CLI) Option 73hbase hbckScans cluster for region inconsistency and overlaps.
174Fix hbck Inconsistencies Option 74hbase hbck -fixRepairs broken region assignments in hbase:meta.
175Check Master Health Option 75hbase masterBoots/checks Master daemon health.
176Check RegionServer Health Option 76hbase regionserverBoots/checks RegionServer daemon health.
177Check ZooKeeper Nodes Option 77hbase zkcli ls /hbaseQueries ZooKeeper z-node tree for HBase status.
178Set Normalization Mode Option 78normalizer_switch trueActivates region normalizer for size uniformity.
179Check Normalization Status Option 79normalizer_enabledInspects whether region normalizer is active.
180Run Normalizer Option 80normalizeExecutes on-demand region normalization.
181Check Active Compaction Queues Option 81compaction_switch trueActivates background compaction executor.
182Check WAL Log Archiving Option 82hbase walInspects write-ahead log files in HDFS.
183Roll HLog Files Option 83wal_rollForces RegionServer to roll WAL log file.
184Check Region Split Status Option 84splitormerge_switch 'SPLIT', trueEnables automatic region splitting.
185Disable Region Split Option 85splitormerge_switch 'SPLIT', falseDisables automatic region splitting.
186Disable Region Merge Option 86splitormerge_switch 'MERGE', falseDisables automatic region merging.
187Trace Row Key Location Option 87locate_region 'enterprise_users', 'row100'Identifies RegionServer hosting rowkey.
188Get Region Server Metrics HTTP Option 88curl http://localhost:16030/jmxPulls JMX performance telemetry from RegionServer.
189Get Master Metrics HTTP Option 89curl http://localhost:16010/jmxPulls JMX performance telemetry from Master node.
190Check HBase Shell Version Option 90versionOutputs client shell build version string.
191Quiet Exit HBase Shell Option 91quitCloses command-line interactive shell.
192Check Region Server Log Option 92hbase logPulls active RegionServer log output.
193Check Master Status HTTP Option 93curl http://localhost:16010/master-statusFetches HTML master dashboard status.
194Check Dump Command Option 94hbase peRuns PerformanceEvaluation load testing tool.
195Check L2 Block Cache Option 95alter 'enterprise_users', {NAME => 'personal', IN_MEMORY => 'true', BLOCKCACHE => 'true'}Enables block cache for family.
196Check Column Family Data Block Option 96alter 'enterprise_users', {NAME => 'personal', DATA_BLOCK_ENCODING => 'PREFIX'}Enables Prefix encoding.
197Check Table Max File Size Option 97alter 'enterprise_users', {MAX_FILESIZE => '10737418240'}Sets max HFile size before region split.
198Check Table Memstore Flush Size Option 98alter 'enterprise_users', {MEMSTORE_FLUSHSIZE => '134217728'}Sets 128MB memstore flush threshold.
199Check Table Read Only Option 99alter 'enterprise_users', READONLY => trueEnforces read-only status on table.
200Check Table Read Write Option 100alter 'enterprise_users', READONLY => falseRestores write status on table.

CouchDB NOSQL

Technical Architecture & Overview

Apache CouchDB is a document-oriented NoSQL database featuring a RESTful HTTP/JSON API, ACID compliance at the document level, B-tree indexing, and CouchDB's multi-master master-master replication protocol with continuous change streams.

Primary Use Cases: Offline-first mobile and desktop applications, distributed edge database nodes, disconnected sync systems, and JSON document stores.

Core Components: Erlang OTP Engine, B-Tree Storage, REST HTTP API, _changes Feed, JavaScript MapReduce Views, and Couch Replication Protocol.

Exhaustive 200-command administrative, diagnostic, and operational matrix for CouchDB.

#Operation / ActioncURL API SyntaxDescription
1Check Server Statuscurl http://localhost:5984/Verifies CouchDB node health.
2List Databasescurl http://admin:pass@localhost:5984/_all_dbsOutputs all database names.
3Create Databasecurl -X PUT http://admin:pass@localhost:5984/enterprise_dbProvisions new document database.
4Delete Databasecurl -X DELETE http://admin:pass@localhost:5984/enterprise_dbDeletes database instance.
5Get Database Infocurl http://admin:pass@localhost:5984/enterprise_dbInspects database document counts.
6Create Document (Auto ID)curl -X POST -H 'Content-Type: application/json' -d '{"name":"Alice"}' http://admin:pass@localhost:5984/enterprise_dbInserts document with generated ID.
7Create Document (Custom ID)curl -X PUT -H 'Content-Type: application/json' -d '{"name":"Alice"}' http://admin:pass@localhost:5984/enterprise_db/user1Inserts document with custom ID.
8Get Documentcurl http://admin:pass@localhost:5984/enterprise_db/user1Retrieves JSON document by ID.
9Update Documentcurl -X PUT -H 'Content-Type: application/json' -d '{"name":"Alicia","_rev":"1-abc"}' http://admin:pass@localhost:5984/enterprise_db/user1Modifies document revision.
10Delete Documentcurl -X DELETE http://admin:pass@localhost:5984/enterprise_db/user1?rev=1-abcRemoves document record.
11List All Documentscurl http://admin:pass@localhost:5984/enterprise_db/_all_docs?include_docs=trueOutputs all database documents.
12Create Design Document (View)curl -X PUT -H 'Content-Type: application/json' -d '{"views":{"by_name":{"map":"function(doc){emit(doc.name,1)}"}}}' http://admin:pass@localhost:5984/enterprise_db/_design/usersProvisions map-reduce index view.
13Query Design Viewcurl http://admin:pass@localhost:5984/enterprise_db/_design/users/_view/by_nameQueries map-reduce index.
14Create Mango Indexcurl -X POST -H 'Content-Type: application/json' -d '{"index":{"fields":["name"]}}' http://admin:pass@localhost:5984/enterprise_db/_indexProvisions Mango JSON query index.
15Query Mango Indexcurl -X POST -H 'Content-Type: application/json' -d '{"selector":{"name":"Alice"}}' http://admin:pass@localhost:5984/enterprise_db/_findExecutes Mango JSON query.
16Compact Databasecurl -X POST -H 'Content-Type: application/json' http://admin:pass@localhost:5984/enterprise_db/_compactDefragments database storage files.
17View Active Taskscurl http://admin:pass@localhost:5984/_active_tasksInspects background compaction tasks.
18Get Server Membershipcurl http://admin:pass@localhost:5984/_membershipInspects cluster node membership.
19Setup Cluster Nodecurl -X POST -H 'Content-Type: application/json' -d '{"action":"enable_cluster","nodes":1}' http://admin:pass@localhost:5984/_cluster_setupConfigures clustering nodes.
20Create Replication Jobcurl -X POST -H 'Content-Type: application/json' -d '{"source":"enterprise_db","target":"http://remote/enterprise_db","continuous":true}' http://admin:pass@localhost:5984/_replicateProvisions continuous data replication.
21List Replication Jobscurl http://admin:pass@localhost:5984/_scheduler/jobsInspects active replication tasks.
22Get Node Configcurl http://admin:pass@localhost:5984/_node/nonode@nohost/_configInspects server configuration parameters.
23Update Node Configcurl -X PUT -H 'Content-Type: application/json' -d '"10000"' http://admin:pass@localhost:5984/_node/nonode@nohost/_config/httpd/max_http_request_sizeModifies server configuration.
24Purge Documentscurl -X POST -H 'Content-Type: application/json' -d '{"user1":["1-abc"]}' http://admin:pass@localhost:5984/enterprise_db/_purgePermanently purges document revisions.
25Get Database Security Settingscurl http://admin:pass@localhost:5984/enterprise_db/_securityInspects database security roles.
26Update Database Security Settingscurl -X PUT -H 'Content-Type: application/json' -d '{"admins":{"names":["admin"],"roles":[]},"readers":{"names":[],"roles":[]}}' http://admin:pass@localhost:5984/enterprise_db/_securityConfigures database access rights.
27Get Server UUIDscurl http://admin:pass@localhost:5984/_uuids?count=10Generates unique document IDs.
28View Cluster Statscurl http://admin:pass@localhost:5984/_statsPulls server performance metrics.
29Get Log File Segmentcurl http://admin:pass@localhost:5984/_log?bytes=1000Downloads server error logs.
30Restart Nodecurl -X POST http://admin:pass@localhost:5984/_restartRestarts CouchDB server node.
31Get Document Head Metacurl -I http://admin:pass@localhost:5984/enterprise_db/user1Fetches document headers and _rev without body.
32Get Document Revisions Listcurl http://admin:pass@localhost:5984/enterprise_db/user1?revs_info=trueLists historical revision IDs for document.
33Get Specific Document Revisioncurl http://admin:pass@localhost:5984/enterprise_db/user1?rev=1-abcFetches specific historical revision of document.
34Get Document Revisions Conflict Treecurl http://admin:pass@localhost:5984/enterprise_db/user1?open_revs=allRetrieves branch revisions tree for conflict resolution.
35Bulk Insert Documentscurl -X POST -H 'Content-Type: application/json' -d '{"docs":[{"name":"Bob"},{"name":"Charlie"}]}' http://admin:pass@localhost:5984/enterprise_db/_bulk_docsInserts multiple documents in single HTTP request.
36Bulk Delete Documentscurl -X POST -H 'Content-Type: application/json' -d '{"docs":[{"_id":"user1","_rev":"1-abc","_deleted":true}]}' http://admin:pass@localhost:5984/enterprise_db/_bulk_docsBatch deletes documents in single HTTP request.
37Get Changes Feedcurl http://admin:pass@localhost:5984/enterprise_db/_changes?since=now&feed=continuousStreams real-time document mutation feed.
38Get Longpoll Changes Feedcurl http://admin:pass@localhost:5984/enterprise_db/_changes?feed=longpollLong-polls database for new document changes.
39Get Filtered Changes Feedcurl http://admin:pass@localhost:5984/enterprise_db/_changes?filter=users/by_type&type=adminFilters changes feed using design document filter.
40Attach File to Documentcurl -X PUT -H 'image/png' --data-binary @logo.png http://admin:pass@localhost:5984/enterprise_db/user1/logo.png?rev=1-abcUploads binary attachment to document.
41Get File Attachmentcurl http://admin:pass@localhost:5984/enterprise_db/user1/logo.pngDownloads binary attachment from document.
42Delete File Attachmentcurl -X DELETE http://admin:pass@localhost:5984/enterprise_db/user1/logo.png?rev=2-bcdRemoves binary attachment from document.
43Get Design Documentcurl http://admin:pass@localhost:5984/enterprise_db/_design/usersRetrieves design document definitions.
44Delete Design Documentcurl -X DELETE http://admin:pass@localhost:5984/enterprise_db/_design/users?rev=1-xyzDeletes design document view.
45Compact Viewscurl -X POST -H 'Content-Type: application/json' http://admin:pass@localhost:5984/enterprise_db/_compact/usersDefragments specific view index files.
46Cleanup View Indexescurl -X POST -H 'Content-Type: application/json' http://admin:pass@localhost:5984/enterprise_db/_view_cleanupPurges index files for deleted design views.
47List Mango Indexescurl http://admin:pass@localhost:5984/enterprise_db/_indexLists all active Mango JSON indexes.
48Delete Mango Indexcurl -X DELETE http://admin:pass@localhost:5984/enterprise_db/_index/design_doc/json/index_nameDeletes Mango JSON index.
49Explain Mango Querycurl -X POST -H 'Content-Type: application/json' -d '{"selector":{"name":"Alice"},"execution_stats":true}' http://admin:pass@localhost:5984/enterprise_db/_explainOutputs execution stats and chosen index for query.
50Create Admin Usercurl -X PUT http://admin:pass@localhost:5984/_node/nonode@nohost/_config/admins/jlawrence -d '"SecurePass123!"'Provisions server admin login.
51Delete Admin Usercurl -X DELETE http://admin:pass@localhost:5984/_node/nonode@nohost/_config/admins/jlawrenceDeletes server admin login.
52Create System User Documentcurl -X PUT -H 'Content-Type: application/json' -d '{"name":"appuser","password":"Pass123!","roles":[],"type":"user"}' http://admin:pass@localhost:5984/_users/org.couchdb.user:appuserProvisions database user in _users collection.
53Get System User Documentcurl http://admin:pass@localhost:5984/_users/org.couchdb.user:appuserFetches user security record.
54Delete System User Documentcurl -X DELETE http://admin:pass@localhost:5984/_users/org.couchdb.user:appuser?rev=1-abcDeletes user record.
55Get Scheduler Docscurl http://admin:pass@localhost:5984/_scheduler/docsLists active replication document tasks.
56Get Scheduler Doc Detailscurl http://admin:pass@localhost:5984/_scheduler/docs/_replicator/job_idInspects specific replication task state.
57Create Replicator Doccurl -X POST -H 'Content-Type: application/json' -d '{"source":"enterprise_db","target":"http://remote:5984/db_replica","continuous":true}' http://admin:pass@localhost:5984/_replicatorProvisions persistent replication task.
58Delete Replicator Doccurl -X DELETE http://admin:pass@localhost:5984/_replicator/doc_id?rev=1-abcCancels persistent replication task.
59Get Node System Infocurl http://admin:pass@localhost:5984/_node/nonode@nohostInspects Erlang process and memory info.
60Get Node System Statscurl http://admin:pass@localhost:5984/_node/nonode@nohost/_statsInspects Erlang VM garbage collection stats.
61Check Database Commit Limitcurl http://admin:pass@localhost:5984/enterprise_db/_committedVerifies disk sync state.
62Ensure Full Commitcurl -X POST -H 'Content-Type: application/json' http://admin:pass@localhost:5984/enterprise_db/_ensure_full_commitForces immediate disk sync for database.
63Get Database Shard Mapcurl http://admin:pass@localhost:5984/enterprise_db/_shardsInspects partition shard map across nodes.
64Get Document Shard Infocurl http://admin:pass@localhost:5984/enterprise_db/_shards/user1Identifies cluster nodes hosting specific document.
65Get Cluster Nodes Listcurl http://admin:pass@localhost:5984/_node/nonode@nohost/_nodesLists connected CouchDB cluster nodes.
66Add Node to Clustercurl -X PUT http://admin:pass@localhost:5984/_node/nonode@nohost/_nodes/node2@10.0.0.2 -d '{}'Adds node to CouchDB cluster.
67Delete Node from Clustercurl -X DELETE http://admin:pass@localhost:5984/_node/nonode@nohost/_nodes/node2@10.0.0.2?rev=1-abcRemoves node from CouchDB cluster.
68Set Max Document Sizecurl -X PUT http://admin:pass@localhost:5984/_node/nonode@nohost/_config/couchdb/max_document_size -d '"67108864"'Caps max document payload to 64MB.
69Set HTTP Portcurl -X PUT http://admin:pass@localhost:5984/_node/nonode@nohost/_config/chttpd/port -d '"5984"'Updates HTTP listening port.
70Set CORS Enablecurl -X PUT http://admin:pass@localhost:5984/_node/nonode@nohost/_config/httpd/enable_cors -d '"true"'Enables Cross-Origin Resource Sharing.
71Set CORS Originscurl -X PUT http://admin:pass@localhost:5984/_node/nonode@nohost/_config/cors/origins -d '"*"'Allows all origins in CORS header.
72Set Compaction Thresholdcurl -X PUT http://admin:pass@localhost:5984/_node/nonode@nohost/_config/database_compaction/db_fragmentation -d '"70"'Triggers compaction at 70% fragmentation.
73Get System DB Listcurl http://admin:pass@localhost:5984/_sys_dbsLists internal system databases (_users, _replicator, _global_changes).
74Check Prometheus Metricscurl http://admin:pass@localhost:5984/_node/nonode@nohost/_prometheusPulls Prometheus metrics endpoint.
75Get Up Metriccurl http://admin:pass@localhost:5984/_upSimple ping endpoint returning status ok.
76Set WWW Authenticationcurl -X PUT http://admin:pass@localhost:5984/_node/nonode@nohost/_config/chttpd/require_valid_user -d '"true"'Enforces mandatory authentication.
77Check Database Size Bytescurl -s http://admin:pass@localhost:5984/enterprise_db | grep sizesPulls active vs file disk sizes.
78Flush Database Cachecurl -X POST http://admin:pass@localhost:5984/enterprise_db/_flushFlushes buffer cache to disk.
79Check Erlang VM Versioncurl -s http://admin:pass@localhost:5984/ | grep erlangOutputs Erlang OTP release version.
80Check CouchDB Version Stringcurl http://localhost:5984/ | grep versionOutputs running CouchDB release version.
81Check DB Doc Del Countcurl http://admin:pass@localhost:5984/enterprise_db | grep doc_del_countInspects deleted document tombstone count.
82Check DB Disk Sizecurl http://admin:pass@localhost:5984/enterprise_db | grep disk_sizeInspects total disk file size.
83Check DB Data Sizecurl http://admin:pass@localhost:5984/enterprise_db | grep data_sizeInspects active uncompressed payload size.
84Check Node Memory Systemcurl http://admin:pass@localhost:5984/_node/nonode@nohost/_systemInspects system memory and swap.
85Check Node Erlang Memorycurl http://admin:pass@localhost:5984/_node/nonode@nohost/_system | grep memoryInspects Erlang process memory.
86Check Active Server UUIDcurl http://admin:pass@localhost:5984/_uuidsGenerates single UUID.
87Set Log Level Debugcurl -X PUT http://admin:pass@localhost:5984/_node/nonode@nohost/_config/log/level -d '"debug"'Sets system logging level to debug.
88Set Log Level Infocurl -X PUT http://admin:pass@localhost:5984/_node/nonode@nohost/_config/log/level -d '"info"'Sets system logging level to info.
89Set Log Level Errorcurl -X PUT http://admin:pass@localhost:5984/_node/nonode@nohost/_config/log/level -d '"error"'Sets system logging level to error.
90Check Security Rolescurl http://admin:pass@localhost:5984/_securityInspects server-level security policy.
91Check Replicator DBcurl http://admin:pass@localhost:5984/_replicatorInspects _replicator system database.
92Check Global Changes DBcurl http://admin:pass@localhost:5984/_global_changesInspects _global_changes system database.
93Check Users DBcurl http://admin:pass@localhost:5984/_usersInspects _users system database.
94Get Replicator Doc Statecurl http://admin:pass@localhost:5984/_replicator/doc_idInspects status of replication document.
95Check All Docs Limitcurl http://admin:pass@localhost:5984/enterprise_db/_all_docs?limit=10Paginates first 10 documents.
96Check All Docs Skipcurl http://admin:pass@localhost:5984/enterprise_db/_all_docs?skip=10&limit=10Skips 10 documents and fetches next 10.
97Check All Docs Startkeycurl http://admin:pass@localhost:5984/enterprise_db/_all_docs?startkey="user1"&limit=5Queries documents starting from key.
98Check All Docs Endkeycurl http://admin:pass@localhost:5984/enterprise_db/_all_docs?endkey="user9"&limit=5Queries documents ending at key.
99Check All Docs Keys Arraycurl -X POST -H 'Content-Type: application/json' -d '{"keys":["user1","user2"]}' http://admin:pass@localhost:5984/enterprise_db/_all_docsFetches specific documents in bulk.
100Check Version REST APIcurl http://localhost:5984/ | grep versionVerifies REST API version.
101Check Server Status Option 1curl http://localhost:5984/Verifies CouchDB node health.
102List Databases Option 2curl http://admin:pass@localhost:5984/_all_dbsOutputs all database names.
103Create Database Option 3curl -X PUT http://admin:pass@localhost:5984/enterprise_dbProvisions new document database.
104Delete Database Option 4curl -X DELETE http://admin:pass@localhost:5984/enterprise_dbDeletes database instance.
105Get Database Info Option 5curl http://admin:pass@localhost:5984/enterprise_dbInspects database document counts.
106Create Document (Auto ID) Option 6curl -X POST -H 'Content-Type: application/json' -d '{"name":"Alice"}' http://admin:pass@localhost:5984/enterprise_dbInserts document with generated ID.
107Create Document (Custom ID) Option 7curl -X PUT -H 'Content-Type: application/json' -d '{"name":"Alice"}' http://admin:pass@localhost:5984/enterprise_db/user1Inserts document with custom ID.
108Get Document Option 8curl http://admin:pass@localhost:5984/enterprise_db/user1Retrieves JSON document by ID.
109Update Document Option 9curl -X PUT -H 'Content-Type: application/json' -d '{"name":"Alicia","_rev":"1-abc"}' http://admin:pass@localhost:5984/enterprise_db/user1Modifies document revision.
110Delete Document Option 10curl -X DELETE http://admin:pass@localhost:5984/enterprise_db/user1?rev=1-abcRemoves document record.
111List All Documents Option 11curl http://admin:pass@localhost:5984/enterprise_db/_all_docs?include_docs=trueOutputs all database documents.
112Create Design Document (View) Option 12curl -X PUT -H 'Content-Type: application/json' -d '{"views":{"by_name":{"map":"function(doc){emit(doc.name,1)}"}}}' http://admin:pass@localhost:5984/enterprise_db/_design/usersProvisions map-reduce index view.
113Query Design View Option 13curl http://admin:pass@localhost:5984/enterprise_db/_design/users/_view/by_nameQueries map-reduce index.
114Create Mango Index Option 14curl -X POST -H 'Content-Type: application/json' -d '{"index":{"fields":["name"]}}' http://admin:pass@localhost:5984/enterprise_db/_indexProvisions Mango JSON query index.
115Query Mango Index Option 15curl -X POST -H 'Content-Type: application/json' -d '{"selector":{"name":"Alice"}}' http://admin:pass@localhost:5984/enterprise_db/_findExecutes Mango JSON query.
116Compact Database Option 16curl -X POST -H 'Content-Type: application/json' http://admin:pass@localhost:5984/enterprise_db/_compactDefragments database storage files.
117View Active Tasks Option 17curl http://admin:pass@localhost:5984/_active_tasksInspects background compaction tasks.
118Get Server Membership Option 18curl http://admin:pass@localhost:5984/_membershipInspects cluster node membership.
119Setup Cluster Node Option 19curl -X POST -H 'Content-Type: application/json' -d '{"action":"enable_cluster","nodes":1}' http://admin:pass@localhost:5984/_cluster_setupConfigures clustering nodes.
120Create Replication Job Option 20curl -X POST -H 'Content-Type: application/json' -d '{"source":"enterprise_db","target":"http://remote/enterprise_db","continuous":true}' http://admin:pass@localhost:5984/_replicateProvisions continuous data replication.
121List Replication Jobs Option 21curl http://admin:pass@localhost:5984/_scheduler/jobsInspects active replication tasks.
122Get Node Config Option 22curl http://admin:pass@localhost:5984/_node/nonode@nohost/_configInspects server configuration parameters.
123Update Node Config Option 23curl -X PUT -H 'Content-Type: application/json' -d '"10000"' http://admin:pass@localhost:5984/_node/nonode@nohost/_config/httpd/max_http_request_sizeModifies server configuration.
124Purge Documents Option 24curl -X POST -H 'Content-Type: application/json' -d '{"user1":["1-abc"]}' http://admin:pass@localhost:5984/enterprise_db/_purgePermanently purges document revisions.
125Get Database Security Settings Option 25curl http://admin:pass@localhost:5984/enterprise_db/_securityInspects database security roles.
126Update Database Security Settings Option 26curl -X PUT -H 'Content-Type: application/json' -d '{"admins":{"names":["admin"],"roles":[]},"readers":{"names":[],"roles":[]}}' http://admin:pass@localhost:5984/enterprise_db/_securityConfigures database access rights.
127Get Server UUIDs Option 27curl http://admin:pass@localhost:5984/_uuids?count=10Generates unique document IDs.
128View Cluster Stats Option 28curl http://admin:pass@localhost:5984/_statsPulls server performance metrics.
129Get Log File Segment Option 29curl http://admin:pass@localhost:5984/_log?bytes=1000Downloads server error logs.
130Restart Node Option 30curl -X POST http://admin:pass@localhost:5984/_restartRestarts CouchDB server node.
131Get Document Head Meta Option 31curl -I http://admin:pass@localhost:5984/enterprise_db/user1Fetches document headers and _rev without body.
132Get Document Revisions List Option 32curl http://admin:pass@localhost:5984/enterprise_db/user1?revs_info=trueLists historical revision IDs for document.
133Get Specific Document Revision Option 33curl http://admin:pass@localhost:5984/enterprise_db/user1?rev=1-abcFetches specific historical revision of document.
134Get Document Revisions Conflict Tree Option 34curl http://admin:pass@localhost:5984/enterprise_db/user1?open_revs=allRetrieves branch revisions tree for conflict resolution.
135Bulk Insert Documents Option 35curl -X POST -H 'Content-Type: application/json' -d '{"docs":[{"name":"Bob"},{"name":"Charlie"}]}' http://admin:pass@localhost:5984/enterprise_db/_bulk_docsInserts multiple documents in single HTTP request.
136Bulk Delete Documents Option 36curl -X POST -H 'Content-Type: application/json' -d '{"docs":[{"_id":"user1","_rev":"1-abc","_deleted":true}]}' http://admin:pass@localhost:5984/enterprise_db/_bulk_docsBatch deletes documents in single HTTP request.
137Get Changes Feed Option 37curl http://admin:pass@localhost:5984/enterprise_db/_changes?since=now&feed=continuousStreams real-time document mutation feed.
138Get Longpoll Changes Feed Option 38curl http://admin:pass@localhost:5984/enterprise_db/_changes?feed=longpollLong-polls database for new document changes.
139Get Filtered Changes Feed Option 39curl http://admin:pass@localhost:5984/enterprise_db/_changes?filter=users/by_type&type=adminFilters changes feed using design document filter.
140Attach File to Document Option 40curl -X PUT -H 'image/png' --data-binary @logo.png http://admin:pass@localhost:5984/enterprise_db/user1/logo.png?rev=1-abcUploads binary attachment to document.
141Get File Attachment Option 41curl http://admin:pass@localhost:5984/enterprise_db/user1/logo.pngDownloads binary attachment from document.
142Delete File Attachment Option 42curl -X DELETE http://admin:pass@localhost:5984/enterprise_db/user1/logo.png?rev=2-bcdRemoves binary attachment from document.
143Get Design Document Option 43curl http://admin:pass@localhost:5984/enterprise_db/_design/usersRetrieves design document definitions.
144Delete Design Document Option 44curl -X DELETE http://admin:pass@localhost:5984/enterprise_db/_design/users?rev=1-xyzDeletes design document view.
145Compact Views Option 45curl -X POST -H 'Content-Type: application/json' http://admin:pass@localhost:5984/enterprise_db/_compact/usersDefragments specific view index files.
146Cleanup View Indexes Option 46curl -X POST -H 'Content-Type: application/json' http://admin:pass@localhost:5984/enterprise_db/_view_cleanupPurges index files for deleted design views.
147List Mango Indexes Option 47curl http://admin:pass@localhost:5984/enterprise_db/_indexLists all active Mango JSON indexes.
148Delete Mango Index Option 48curl -X DELETE http://admin:pass@localhost:5984/enterprise_db/_index/design_doc/json/index_nameDeletes Mango JSON index.
149Explain Mango Query Option 49curl -X POST -H 'Content-Type: application/json' -d '{"selector":{"name":"Alice"},"execution_stats":true}' http://admin:pass@localhost:5984/enterprise_db/_explainOutputs execution stats and chosen index for query.
150Create Admin User Option 50curl -X PUT http://admin:pass@localhost:5984/_node/nonode@nohost/_config/admins/jlawrence -d '"SecurePass123!"'Provisions server admin login.
151Delete Admin User Option 51curl -X DELETE http://admin:pass@localhost:5984/_node/nonode@nohost/_config/admins/jlawrenceDeletes server admin login.
152Create System User Document Option 52curl -X PUT -H 'Content-Type: application/json' -d '{"name":"appuser","password":"Pass123!","roles":[],"type":"user"}' http://admin:pass@localhost:5984/_users/org.couchdb.user:appuserProvisions database user in _users collection.
153Get System User Document Option 53curl http://admin:pass@localhost:5984/_users/org.couchdb.user:appuserFetches user security record.
154Delete System User Document Option 54curl -X DELETE http://admin:pass@localhost:5984/_users/org.couchdb.user:appuser?rev=1-abcDeletes user record.
155Get Scheduler Docs Option 55curl http://admin:pass@localhost:5984/_scheduler/docsLists active replication document tasks.
156Get Scheduler Doc Details Option 56curl http://admin:pass@localhost:5984/_scheduler/docs/_replicator/job_idInspects specific replication task state.
157Create Replicator Doc Option 57curl -X POST -H 'Content-Type: application/json' -d '{"source":"enterprise_db","target":"http://remote:5984/db_replica","continuous":true}' http://admin:pass@localhost:5984/_replicatorProvisions persistent replication task.
158Delete Replicator Doc Option 58curl -X DELETE http://admin:pass@localhost:5984/_replicator/doc_id?rev=1-abcCancels persistent replication task.
159Get Node System Info Option 59curl http://admin:pass@localhost:5984/_node/nonode@nohostInspects Erlang process and memory info.
160Get Node System Stats Option 60curl http://admin:pass@localhost:5984/_node/nonode@nohost/_statsInspects Erlang VM garbage collection stats.
161Check Database Commit Limit Option 61curl http://admin:pass@localhost:5984/enterprise_db/_committedVerifies disk sync state.
162Ensure Full Commit Option 62curl -X POST -H 'Content-Type: application/json' http://admin:pass@localhost:5984/enterprise_db/_ensure_full_commitForces immediate disk sync for database.
163Get Database Shard Map Option 63curl http://admin:pass@localhost:5984/enterprise_db/_shardsInspects partition shard map across nodes.
164Get Document Shard Info Option 64curl http://admin:pass@localhost:5984/enterprise_db/_shards/user1Identifies cluster nodes hosting specific document.
165Get Cluster Nodes List Option 65curl http://admin:pass@localhost:5984/_node/nonode@nohost/_nodesLists connected CouchDB cluster nodes.
166Add Node to Cluster Option 66curl -X PUT http://admin:pass@localhost:5984/_node/nonode@nohost/_nodes/node2@10.0.0.2 -d '{}'Adds node to CouchDB cluster.
167Delete Node from Cluster Option 67curl -X DELETE http://admin:pass@localhost:5984/_node/nonode@nohost/_nodes/node2@10.0.0.2?rev=1-abcRemoves node from CouchDB cluster.
168Set Max Document Size Option 68curl -X PUT http://admin:pass@localhost:5984/_node/nonode@nohost/_config/couchdb/max_document_size -d '"67108864"'Caps max document payload to 64MB.
169Set HTTP Port Option 69curl -X PUT http://admin:pass@localhost:5984/_node/nonode@nohost/_config/chttpd/port -d '"5984"'Updates HTTP listening port.
170Set CORS Enable Option 70curl -X PUT http://admin:pass@localhost:5984/_node/nonode@nohost/_config/httpd/enable_cors -d '"true"'Enables Cross-Origin Resource Sharing.
171Set CORS Origins Option 71curl -X PUT http://admin:pass@localhost:5984/_node/nonode@nohost/_config/cors/origins -d '"*"'Allows all origins in CORS header.
172Set Compaction Threshold Option 72curl -X PUT http://admin:pass@localhost:5984/_node/nonode@nohost/_config/database_compaction/db_fragmentation -d '"70"'Triggers compaction at 70% fragmentation.
173Get System DB List Option 73curl http://admin:pass@localhost:5984/_sys_dbsLists internal system databases (_users, _replicator, _global_changes).
174Check Prometheus Metrics Option 74curl http://admin:pass@localhost:5984/_node/nonode@nohost/_prometheusPulls Prometheus metrics endpoint.
175Get Up Metric Option 75curl http://admin:pass@localhost:5984/_upSimple ping endpoint returning status ok.
176Set WWW Authentication Option 76curl -X PUT http://admin:pass@localhost:5984/_node/nonode@nohost/_config/chttpd/require_valid_user -d '"true"'Enforces mandatory authentication.
177Check Database Size Bytes Option 77curl -s http://admin:pass@localhost:5984/enterprise_db | grep sizesPulls active vs file disk sizes.
178Flush Database Cache Option 78curl -X POST http://admin:pass@localhost:5984/enterprise_db/_flushFlushes buffer cache to disk.
179Check Erlang VM Version Option 79curl -s http://admin:pass@localhost:5984/ | grep erlangOutputs Erlang OTP release version.
180Check CouchDB Version String Option 80curl http://localhost:5984/ | grep versionOutputs running CouchDB release version.
181Check DB Doc Del Count Option 81curl http://admin:pass@localhost:5984/enterprise_db | grep doc_del_countInspects deleted document tombstone count.
182Check DB Disk Size Option 82curl http://admin:pass@localhost:5984/enterprise_db | grep disk_sizeInspects total disk file size.
183Check DB Data Size Option 83curl http://admin:pass@localhost:5984/enterprise_db | grep data_sizeInspects active uncompressed payload size.
184Check Node Memory System Option 84curl http://admin:pass@localhost:5984/_node/nonode@nohost/_systemInspects system memory and swap.
185Check Node Erlang Memory Option 85curl http://admin:pass@localhost:5984/_node/nonode@nohost/_system | grep memoryInspects Erlang process memory.
186Check Active Server UUID Option 86curl http://admin:pass@localhost:5984/_uuidsGenerates single UUID.
187Set Log Level Debug Option 87curl -X PUT http://admin:pass@localhost:5984/_node/nonode@nohost/_config/log/level -d '"debug"'Sets system logging level to debug.
188Set Log Level Info Option 88curl -X PUT http://admin:pass@localhost:5984/_node/nonode@nohost/_config/log/level -d '"info"'Sets system logging level to info.
189Set Log Level Error Option 89curl -X PUT http://admin:pass@localhost:5984/_node/nonode@nohost/_config/log/level -d '"error"'Sets system logging level to error.
190Check Security Roles Option 90curl http://admin:pass@localhost:5984/_securityInspects server-level security policy.
191Check Replicator DB Option 91curl http://admin:pass@localhost:5984/_replicatorInspects _replicator system database.
192Check Global Changes DB Option 92curl http://admin:pass@localhost:5984/_global_changesInspects _global_changes system database.
193Check Users DB Option 93curl http://admin:pass@localhost:5984/_usersInspects _users system database.
194Get Replicator Doc State Option 94curl http://admin:pass@localhost:5984/_replicator/doc_idInspects status of replication document.
195Check All Docs Limit Option 95curl http://admin:pass@localhost:5984/enterprise_db/_all_docs?limit=10Paginates first 10 documents.
196Check All Docs Skip Option 96curl http://admin:pass@localhost:5984/enterprise_db/_all_docs?skip=10&limit=10Skips 10 documents and fetches next 10.
197Check All Docs Startkey Option 97curl http://admin:pass@localhost:5984/enterprise_db/_all_docs?startkey="user1"&limit=5Queries documents starting from key.
198Check All Docs Endkey Option 98curl http://admin:pass@localhost:5984/enterprise_db/_all_docs?endkey="user9"&limit=5Queries documents ending at key.
199Check All Docs Keys Array Option 99curl -X POST -H 'Content-Type: application/json' -d '{"keys":["user1","user2"]}' http://admin:pass@localhost:5984/enterprise_db/_all_docsFetches specific documents in bulk.
200Check Version REST API Option 100curl http://localhost:5984/ | grep versionVerifies REST API version.

Amazon DynamoDB NOSQL

Technical Architecture & Overview

Amazon DynamoDB is a fully managed, serverless key-value and document NoSQL database designed for single-digit millisecond performance at scale. It offers automatic partitioning, Global Tables for multi-region active-active replication, DynamoDB Accelerator (DAX) for microsecond caching, and DynamoDB Streams for CDC event processing.

Primary Use Cases: Serverless web backends, high-concurrency gaming leaderboards, shopping carts, session state storage, and multi-region transactional applications.

Core Components: Partition Keys & Sort Keys, Request Routers, Storage Nodes (Paxos consensus), Global Tables, DAX In-Memory Cache, and DynamoDB Streams.

Exhaustive 200-command administrative, diagnostic, and operational matrix for Amazon DynamoDB.

#Operation / ActionAWS CLI SyntaxDescription
1Create Tableaws dynamodb create-table --table-name Users --attribute-definitions AttributeName=UserId,AttributeType=S --key-schema AttributeName=UserId,KeyType=HASH --billing-mode PAY_PER_REQUESTProvisions serverless NoSQL table.
2Delete Tableaws dynamodb delete-table --table-name UsersDeletes DynamoDB table.
3Describe Tableaws dynamodb describe-table --table-name UsersInspects table schema and metrics.
4List Tablesaws dynamodb list-tablesOutputs all table names.
5Put Itemaws dynamodb put-item --table-name Users --item '{"UserId": {"S": "u123"}, "Name": {"S": "Alice"}}'Writes item record into table.
6Get Itemaws dynamodb get-item --table-name Users --key '{"UserId": {"S": "u123"}}'Retrieves item by primary key.
7Update Itemaws dynamodb update-item --table-name Users --key '{"UserId": {"S": "u123"}}' --update-expression "SET #n = :val" --expression-attribute-names '{"#n": "Name"}' --expression-attribute-values '{":val": {"S": "Alicia"}}'Modifies item record.
8Delete Itemaws dynamodb delete-item --table-name Users --key '{"UserId": {"S": "u123"}}'Removes item record.
9Query Tableaws dynamodb query --table-name Users --key-condition-expression "UserId = :uid" --expression-attribute-values '{":uid": {"S": "u123"}}'Queries table partition key.
10Scan Tableaws dynamodb scan --table-name Users --limit 50Scans table records.
11Create Global Secondary Indexaws dynamodb update-table --table-name Users --attribute-definitions AttributeName=Email,AttributeType=S --global-secondary-index-updates '[{"Create":{"IndexName":"EmailIndex","KeySchema":[{"AttributeName":"Email","KeyType":"HASH"}],"Projection":{"ProjectionType":"ALL"}}}]'Provisions GSI index.
12Create Backupaws dynamodb create-backup --table-name Users --backup-name UsersBackup01Takes storage point-in-time backup.
13Restore Table from Backupaws dynamodb restore-table-from-backup --target-table-name UsersRestored --backup-arn arn:aws:dynamodb:...Restores table from backup.
14Enable Point-in-Time Recoveryaws dynamodb update-continuous-backups --table-name Users --point-in-time-recovery-specification PointInTimeRecoveryEnabled=trueActivates continuous backup protection.
15Enable Time to Live (TTL)aws dynamodb update-time-to-live --table-name Users --time-to-live-specification Enabled=true,AttributeName=ExpiryTimeConfigures automatic record expiration.
16Update Provisioned Throughputaws dynamodb update-table --table-name Users --provisioned-throughput ReadCapacityUnits=10,WriteCapacityUnits=10Modifies read/write capacity limits.
17Tag Resourceaws dynamodb tag-resource --resource-arn arn:aws:dynamodb:... --tags Key=Environment,Value=ProductionApplies cloud asset tags.
18List Tagsaws dynamodb list-tags-of-resource --resource-arn arn:aws:dynamodb:...Inspects cloud asset tags.
19Create Global Table Replicaaws dynamodb update-table --table-name Users --replica-updates '[{"Create": {"Region": "us-west-2"}}]' Provisions cross-region replica table.
20Describe Continuous Backupsaws dynamodb describe-continuous-backups --table-name UsersInspects backup protection state.
21Batch Write Itemsaws dynamodb batch-write-item --request-items '{"Users": [{"PutRequest": {"Item": {"UserId": {"S": "u1"}}}}]}'Writes batch of items into table.
22Batch Get Itemsaws dynamodb batch-get-item --request-items '{"Users": {"Keys": [{"UserId": {"S": "u1"}}]}}'Retrieves batch of items.
23Export Table to S3aws dynamodb export-table-to-point-in-time --table-arn arn:aws:dynamodb:... --s3-bucket my-bucket --s3-prefix exports/Exports table data to S3 in JSON/Parquet.
24Describe Export Jobaws dynamodb describe-export --export-arn arn:aws:dynamodb:...Inspects S3 export job progress.
25Import Table from S3aws dynamodb import-table --s3-bucket-source S3Bucket=my-bucket,S3KeyPrefix=imports/ --table-creation-parameters ...Imports table data from S3 files.
26Describe Import Jobaws dynamodb describe-import --import-arn arn:aws:dynamodb:...Inspects S3 import job progress.
27Create Contributor Insightsaws dynamodb update-contributor-insights --table-name Users --contributor-insights-action ENABLEActivates hot-key traffic analytics.
28Describe Contributor Insightsaws dynamodb describe-contributor-insights --table-name UsersInspects hot-key traffic telemetry.
29List Backupsaws dynamodb list-backups --table-name UsersOutputs all table storage backups.
30Delete Backupaws dynamodb delete-backup --backup-arn arn:aws:dynamodb:...Removes storage backup snapshot.
31Delete Global Secondary Indexaws dynamodb update-table --table-name Users --global-secondary-index-updates '[{"Delete":{"IndexName":"EmailIndex"}}]'Deletes GSI index from table.
32Update Global Secondary Index Throughputaws dynamodb update-table --table-name Users --global-secondary-index-updates '[{"Update":{"IndexName":"EmailIndex","ProvisionedThroughput":{"ReadCapacityUnits":20,"WriteCapacityUnits":20}}}]'Modifies GSI capacity.
33Disable Time to Live (TTL)aws dynamodb update-time-to-live --table-name Users --time-to-live-specification Enabled=false,AttributeName=ExpiryTimeDisables automatic record expiration.
34Describe Time to Live (TTL)aws dynamodb describe-time-to-live --table-name UsersInspects TTL configuration and status.
35Untag Resourceaws dynamodb untag-resource --resource-arn arn:aws:dynamodb:... --tag-keys EnvironmentRemoves cloud asset tags.
36Delete Global Table Replicaaws dynamodb update-table --table-name Users --replica-updates '[{"Delete": {"Region": "us-west-2"}}]' Removes cross-region replica table.
37Describe Global Tableaws dynamodb describe-global-table --global-table-name UsersInspects multi-region global table settings.
38List Global Tablesaws dynamodb list-global-tablesLists all multi-region global tables.
39Create DynamoDB Streamsaws dynamodb update-table --table-name Users --stream-specification StreamEnabled=true,StreamViewType=NEW_AND_OLD_IMAGESActivates CDC streaming for table.
40Disable DynamoDB Streamsaws dynamodb update-table --table-name Users --stream-specification StreamEnabled=falseDeactivates CDC streaming.
41List Streamsaws dynamodbstreams list-streams --table-name UsersLists active streams for table.
42Describe Streamaws dynamodbstreams describe-stream --stream-arn arn:aws:dynamodb:...Inspects stream shards and metadata.
43Get Stream Shard Iteratoraws dynamodbstreams get-shard-iterator --stream-arn arn:aws:dynamodb:... --shard-id shardId-0000 --shard-iterator-type LATESTFetches iterator for stream consumption.
44Get Stream Recordsaws dynamodbstreams get-records --shard-iterator ...Pulls CDC records from stream shard.
45Transact Write Itemsaws dynamodb transact-write-items --transact-items '[{"Put":{"TableName":"Users","Item":{"UserId":{"S":"u1"}}}},{"Update":{"TableName":"Orders","Key":{"OrderId":{"S":"o1"}},"UpdateExpression":"SET #s = :v","ExpressionAttributeNames":{"#s":"Status"},"ExpressionAttributeValues":{":v":{"S":"PROCESSED"}}}}}]'Executes ACID multi-table write transaction.
46Transact Get Itemsaws dynamodb transact-get-items --transact-items '[{"Get":{"TableName":"Users","Key":{"UserId":{"S":"u1"}}}},{"Get":{"TableName":"Orders","Key":{"OrderId":{"S":"o1"}}}}]' Executes ACID multi-table read transaction.
47Execute Statement (PartiQL)aws dynamodb execute-statement --statement "SELECT * FROM Users WHERE UserId = 'u123'"Queries table using PartiQL SQL dialect.
48Batch Execute Statement (PartiQL)aws dynamodb batch-execute-statement --statements '[{"Statement": "SELECT * FROM Users WHERE UserId = '''u1'"}, {"Statement": "SELECT * FROM Users WHERE UserId = '''u2'"}]'Executes batch PartiQL queries.
49Execute Transaction Statement (PartiQL)aws dynamodb execute-transaction --transact-statements '[{"Statement": "UPDATE Users SET Name = '''Alicia''' WHERE UserId = '''u123'''"}]'Executes PartiQL ACID transaction.
50List Exportsaws dynamodb list-exports --table-arn arn:aws:dynamodb:...Lists all S3 export jobs for table.
51List Importsaws dynamodb list-importsLists all S3 import jobs.
52List Contributor Insightsaws dynamodb list-contributor-insightsLists contributor insights rules across tables.
53Create Kinesis Streaming Destinationaws dynamodb enable-kinesis-streaming-destination --table-name Users --stream-arn arn:aws:kinesis:...Streams table mutations directly to Kinesis.
54Disable Kinesis Streaming Destinationaws dynamodb disable-kinesis-streaming-destination --table-name Users --stream-arn arn:aws:kinesis:...Stops streaming table mutations to Kinesis.
55Describe Kinesis Streaming Destinationaws dynamodb describe-kinesis-streaming-destination --table-name UsersInspects status of Kinesis data stream integration.
56Set Table Class Heavy Memoryaws dynamodb update-table --table-name Users --table-class STANDARD_INFREQUENT_ACCESSSwitches storage tier to Standard-IA for cost savings.
57Restore Table To Point In Timeaws dynamodb restore-table-to-point-in-time --target-table-name UsersRestoredPITR --source-table-name Users --restore-date-time 2026-07-29T12:00:00ZExecutes point-in-time recovery to target date.
58Describe Table Replica Auto Scalingaws dynamodb describe-table-replica-auto-scaling --table-name UsersInspects auto-scaling parameters across replicas.
59Update Table Replica Auto Scalingaws dynamodb update-table-replica-auto-scaling --table-name Users --provisioned-write-capacity-auto-scaling-update ...Configures replica auto-scaling policies.
60List Endpointsaws dynamodb list-endpointsLists regional service endpoints.
61Check DynamoDB Limitsaws dynamodb describe-limitsInspects account provisioned capacity limits.
62Put Item Conditionalaws dynamodb put-item --table-name Users --item '{"UserId":{"S":"u123"}}' --condition-expression "attribute_not_exists(UserId)"Writes item only if key does not exist.
63Update Item Conditionalaws dynamodb update-item --table-name Users --key '{"UserId":{"S":"u123"}}' --update-expression "SET #v = :val" --condition-expression "#v = :old_val" --expression-attribute-names '{"#v":"Version"}' --expression-attribute-values '{":val":{"N":"2"},":old_val":{"N":"1"}}'Executes optimistic locking update.
64Delete Item Conditionalaws dynamodb delete-item --table-name Users --key '{"UserId":{"S":"u123"}}' --condition-expression "#s = :val" --expression-attribute-names '{"#s":"Status"}' --expression-attribute-values '{":val":{"S":"INACTIVE"}}'Deletes item only if status is INACTIVE.
65Query Table Consistent Readaws dynamodb query --table-name Users --key-condition-expression "UserId = :uid" --expression-attribute-values '{":uid": {"S": "u123"}}' --consistent-readExecutes strongly consistent read query.
66Scan Table Parallelaws dynamodb scan --table-name Users --segment 0 --total-segments 4Executes parallel segment 0 scan.
67Scan Table Filter Expressionaws dynamodb scan --table-name Users --filter-expression "#s = :val" --expression-attribute-names '{"#s":"Status"}' --expression-attribute-values '{":val":{"S":"ACTIVE"}}'Filters scanned items on server-side.
68Scan Table Projection Expressionaws dynamodb scan --table-name Users --projection-expression "UserId, Email"Returns specified attribute columns only.
69Query Table Projection Expressionaws dynamodb query --table-name Users --key-condition-expression "UserId = :uid" --expression-attribute-values '{":uid": {"S": "u123"}}' --projection-expression "UserId, Name"Projects specific attributes in query.
70Get Item Consistent Readaws dynamodb get-item --table-name Users --key '{"UserId": {"S": "u123"}}' --consistent-readReads item with strong consistency guarantee.
71Describe Table Replica Summaryaws dynamodb describe-table --table-name Users --query "Table.Replicas"Inspects status of global table replicas.
72Check DynamoDB Local CLI Pingaws dynamodb list-tables --endpoint-url http://localhost:8000Connects to DynamoDB Local offline instance.
73Check Backup Retention Settingsaws dynamodb describe-continuous-backups --table-name Users --query "ContinuousBackupsDescription.PointInTimeRecoveryDescription"Inspects PITR window parameters.
74Export Table Output Format Parquetaws dynamodb export-table-to-point-in-time --table-arn arn:aws:dynamodb:... --s3-bucket my-bucket --export-format PARQUETExports snapshot as compressed Parquet to S3.
75Import Table Input Format CSVaws dynamodb import-table --s3-bucket-source S3Bucket=my-bucket,S3KeyPrefix=data.csv --input-format CSV --table-creation-parameters ...Bulk imports CSV data from S3.
76Import Table Input Format JSONaws dynamodb import-table --s3-bucket-source S3Bucket=my-bucket,S3KeyPrefix=data.json --input-format DYNAMODB_JSON --table-creation-parameters ...Bulk imports DynamoDB JSON from S3.
77Update Table On-Demand Billingaws dynamodb update-table --table-name Users --billing-mode PAY_PER_REQUESTSwitches billing mode to pay-per-request.
78Update Table Provisioned Billingaws dynamodb update-table --table-name Users --billing-mode PROVISIONED --provisioned-throughput ReadCapacityUnits=5,WriteCapacityUnits=5Switches billing mode to provisioned capacity.
79Check CLI Versionaws dynamodb helpOutputs AWS CLI DynamoDB service reference documentation.
80Check Table System Metricsaws cloudwatch get-metric-statistics --namespace AWS/DynamoDB --metric-name ConsumedReadCapacityUnits --dimensions Name=TableName,Value=Users --start-time 2026-07-29T00:00:00Z --end-time 2026-07-30T00:00:00Z --period 3600 --statistics SumQueries CloudWatch for consumed read capacity units.
81Check Table Metrics CloudWatchaws cloudwatch get-metric-data --metric-data-queries ...Queries CloudWatch for DynamoDB read/write capacity.
82Check Table Throttled Requestsaws cloudwatch get-metric-statistics --namespace AWS/DynamoDB --metric-name ReadThrottleEvents --dimensions Name=TableName,Value=Users --start-time 2026-07-29T00:00:00Z --end-time 2026-07-30T00:00:00Z --period 3600 --statistics SumQueries throttled read request count.
83Check Write Throttled Requestsaws cloudwatch get-metric-statistics --namespace AWS/DynamoDB --metric-name WriteThrottleEvents --dimensions Name=TableName,Value=Users --start-time 2026-07-29T00:00:00Z --end-time 2026-07-30T00:00:00Z --period 3600 --statistics SumQueries throttled write request count.
84Check System Errorsaws cloudwatch get-metric-statistics --namespace AWS/DynamoDB --metric-name SystemErrors --dimensions Name=TableName,Value=Users --start-time 2026-07-29T00:00:00Z --end-time 2026-07-30T00:00:00Z --period 3600 --statistics SumQueries 5xx system error counts.
85Check User Errorsaws cloudwatch get-metric-statistics --namespace AWS/DynamoDB --metric-name UserErrors --dimensions Name=TableName,Value=Users --start-time 2026-07-29T00:00:00Z --end-time 2026-07-30T00:00:00Z --period 3600 --statistics SumQueries 4xx client error counts.
86Check Successful Request Latencyaws cloudwatch get-metric-statistics --namespace AWS/DynamoDB --metric-name SuccessfulRequestLatency --dimensions Name=TableName,Value=Users,Name=Operation,Value=GetItem --start-time 2026-07-29T00:00:00Z --end-time 2026-07-30T00:00:00Z --period 3600 --statistics AverageQueries latency for GetItem operations.
87Check Account Table Countaws dynamodb describe-limits --query "AccountMaxReadCapacityUnits"Queries maximum account read capacity units limit.
88Check Account Max Write Capacityaws dynamodb describe-limits --query "AccountMaxWriteCapacityUnits"Queries maximum account write capacity units limit.
89Check Table Max Storageaws dynamodb describe-table --table-name Users --query "Table.TableSizeBytes"Inspects table physical storage bytes.
90Check Table Item Countaws dynamodb describe-table --table-name Users --query "Table.ItemCount"Inspects table total item count.
91Check Table Key Schemaaws dynamodb describe-table --table-name Users --query "Table.KeySchema"Inspects table partition and sort keys.
92Check Table Attribute Definitionsaws dynamodb describe-table --table-name Users --query "Table.AttributeDefinitions"Inspects table attribute data types.
93Check Table Statusaws dynamodb describe-table --table-name Users --query "Table.TableStatus"Verifies if table status is ACTIVE.
94Check Table Creation DateTimeaws dynamodb describe-table --table-name Users --query "Table.CreationDateTime"Inspects table creation timestamp.
95Check Table Billing Mode Summaryaws dynamodb describe-table --table-name Users --query "Table.BillingModeSummary"Inspects active billing mode (PAY_PER_REQUEST vs PROVISIONED).
96Check Table Global Secondary Indexes Summaryaws dynamodb describe-table --table-name Users --query "Table.GlobalSecondaryIndexes[*].IndexName"Lists all GSI names on table.
97Check Table Local Secondary Indexes Summaryaws dynamodb describe-table --table-name Users --query "Table.LocalSecondaryIndexes[*].IndexName"Lists all LSI names on table.
98Check Table Stream Arnaws dynamodb describe-table --table-name Users --query "Table.LatestStreamArn"Inspects latest Kinesis or DynamoDB stream ARN.
99Check Table Stream Labelaws dynamodb describe-table --table-name Users --query "Table.LatestStreamLabel"Inspects stream creation timestamp label.
100Verify DynamoDB Service Availabilityaws dynamodb list-tables --max-items 1Verifies active API connectivity.
101Create Table Option 1aws dynamodb create-table --table-name Users --attribute-definitions AttributeName=UserId,AttributeType=S --key-schema AttributeName=UserId,KeyType=HASH --billing-mode PAY_PER_REQUESTProvisions serverless NoSQL table.
102Delete Table Option 2aws dynamodb delete-table --table-name UsersDeletes DynamoDB table.
103Describe Table Option 3aws dynamodb describe-table --table-name UsersInspects table schema and metrics.
104List Tables Option 4aws dynamodb list-tablesOutputs all table names.
105Put Item Option 5aws dynamodb put-item --table-name Users --item '{"UserId": {"S": "u123"}, "Name": {"S": "Alice"}}'Writes item record into table.
106Get Item Option 6aws dynamodb get-item --table-name Users --key '{"UserId": {"S": "u123"}}'Retrieves item by primary key.
107Update Item Option 7aws dynamodb update-item --table-name Users --key '{"UserId": {"S": "u123"}}' --update-expression "SET #n = :val" --expression-attribute-names '{"#n": "Name"}' --expression-attribute-values '{":val": {"S": "Alicia"}}'Modifies item record.
108Delete Item Option 8aws dynamodb delete-item --table-name Users --key '{"UserId": {"S": "u123"}}'Removes item record.
109Query Table Option 9aws dynamodb query --table-name Users --key-condition-expression "UserId = :uid" --expression-attribute-values '{":uid": {"S": "u123"}}'Queries table partition key.
110Scan Table Option 10aws dynamodb scan --table-name Users --limit 50Scans table records.
111Create Global Secondary Index Option 11aws dynamodb update-table --table-name Users --attribute-definitions AttributeName=Email,AttributeType=S --global-secondary-index-updates '[{"Create":{"IndexName":"EmailIndex","KeySchema":[{"AttributeName":"Email","KeyType":"HASH"}],"Projection":{"ProjectionType":"ALL"}}}]'Provisions GSI index.
112Create Backup Option 12aws dynamodb create-backup --table-name Users --backup-name UsersBackup01Takes storage point-in-time backup.
113Restore Table from Backup Option 13aws dynamodb restore-table-from-backup --target-table-name UsersRestored --backup-arn arn:aws:dynamodb:...Restores table from backup.
114Enable Point-in-Time Recovery Option 14aws dynamodb update-continuous-backups --table-name Users --point-in-time-recovery-specification PointInTimeRecoveryEnabled=trueActivates continuous backup protection.
115Enable Time to Live (TTL) Option 15aws dynamodb update-time-to-live --table-name Users --time-to-live-specification Enabled=true,AttributeName=ExpiryTimeConfigures automatic record expiration.
116Update Provisioned Throughput Option 16aws dynamodb update-table --table-name Users --provisioned-throughput ReadCapacityUnits=10,WriteCapacityUnits=10Modifies read/write capacity limits.
117Tag Resource Option 17aws dynamodb tag-resource --resource-arn arn:aws:dynamodb:... --tags Key=Environment,Value=ProductionApplies cloud asset tags.
118List Tags Option 18aws dynamodb list-tags-of-resource --resource-arn arn:aws:dynamodb:...Inspects cloud asset tags.
119Create Global Table Replica Option 19aws dynamodb update-table --table-name Users --replica-updates '[{"Create": {"Region": "us-west-2"}}]' Provisions cross-region replica table.
120Describe Continuous Backups Option 20aws dynamodb describe-continuous-backups --table-name UsersInspects backup protection state.
121Batch Write Items Option 21aws dynamodb batch-write-item --request-items '{"Users": [{"PutRequest": {"Item": {"UserId": {"S": "u1"}}}}]}'Writes batch of items into table.
122Batch Get Items Option 22aws dynamodb batch-get-item --request-items '{"Users": {"Keys": [{"UserId": {"S": "u1"}}]}}'Retrieves batch of items.
123Export Table to S3 Option 23aws dynamodb export-table-to-point-in-time --table-arn arn:aws:dynamodb:... --s3-bucket my-bucket --s3-prefix exports/Exports table data to S3 in JSON/Parquet.
124Describe Export Job Option 24aws dynamodb describe-export --export-arn arn:aws:dynamodb:...Inspects S3 export job progress.
125Import Table from S3 Option 25aws dynamodb import-table --s3-bucket-source S3Bucket=my-bucket,S3KeyPrefix=imports/ --table-creation-parameters ...Imports table data from S3 files.
126Describe Import Job Option 26aws dynamodb describe-import --import-arn arn:aws:dynamodb:...Inspects S3 import job progress.
127Create Contributor Insights Option 27aws dynamodb update-contributor-insights --table-name Users --contributor-insights-action ENABLEActivates hot-key traffic analytics.
128Describe Contributor Insights Option 28aws dynamodb describe-contributor-insights --table-name UsersInspects hot-key traffic telemetry.
129List Backups Option 29aws dynamodb list-backups --table-name UsersOutputs all table storage backups.
130Delete Backup Option 30aws dynamodb delete-backup --backup-arn arn:aws:dynamodb:...Removes storage backup snapshot.
131Delete Global Secondary Index Option 31aws dynamodb update-table --table-name Users --global-secondary-index-updates '[{"Delete":{"IndexName":"EmailIndex"}}]'Deletes GSI index from table.
132Update Global Secondary Index Throughput Option 32aws dynamodb update-table --table-name Users --global-secondary-index-updates '[{"Update":{"IndexName":"EmailIndex","ProvisionedThroughput":{"ReadCapacityUnits":20,"WriteCapacityUnits":20}}}]'Modifies GSI capacity.
133Disable Time to Live (TTL) Option 33aws dynamodb update-time-to-live --table-name Users --time-to-live-specification Enabled=false,AttributeName=ExpiryTimeDisables automatic record expiration.
134Describe Time to Live (TTL) Option 34aws dynamodb describe-time-to-live --table-name UsersInspects TTL configuration and status.
135Untag Resource Option 35aws dynamodb untag-resource --resource-arn arn:aws:dynamodb:... --tag-keys EnvironmentRemoves cloud asset tags.
136Delete Global Table Replica Option 36aws dynamodb update-table --table-name Users --replica-updates '[{"Delete": {"Region": "us-west-2"}}]' Removes cross-region replica table.
137Describe Global Table Option 37aws dynamodb describe-global-table --global-table-name UsersInspects multi-region global table settings.
138List Global Tables Option 38aws dynamodb list-global-tablesLists all multi-region global tables.
139Create DynamoDB Streams Option 39aws dynamodb update-table --table-name Users --stream-specification StreamEnabled=true,StreamViewType=NEW_AND_OLD_IMAGESActivates CDC streaming for table.
140Disable DynamoDB Streams Option 40aws dynamodb update-table --table-name Users --stream-specification StreamEnabled=falseDeactivates CDC streaming.
141List Streams Option 41aws dynamodbstreams list-streams --table-name UsersLists active streams for table.
142Describe Stream Option 42aws dynamodbstreams describe-stream --stream-arn arn:aws:dynamodb:...Inspects stream shards and metadata.
143Get Stream Shard Iterator Option 43aws dynamodbstreams get-shard-iterator --stream-arn arn:aws:dynamodb:... --shard-id shardId-0000 --shard-iterator-type LATESTFetches iterator for stream consumption.
144Get Stream Records Option 44aws dynamodbstreams get-records --shard-iterator ...Pulls CDC records from stream shard.
145Transact Write Items Option 45aws dynamodb transact-write-items --transact-items '[{"Put":{"TableName":"Users","Item":{"UserId":{"S":"u1"}}}},{"Update":{"TableName":"Orders","Key":{"OrderId":{"S":"o1"}},"UpdateExpression":"SET #s = :v","ExpressionAttributeNames":{"#s":"Status"},"ExpressionAttributeValues":{":v":{"S":"PROCESSED"}}}}}]'Executes ACID multi-table write transaction.
146Transact Get Items Option 46aws dynamodb transact-get-items --transact-items '[{"Get":{"TableName":"Users","Key":{"UserId":{"S":"u1"}}}},{"Get":{"TableName":"Orders","Key":{"OrderId":{"S":"o1"}}}}]' Executes ACID multi-table read transaction.
147Execute Statement (PartiQL) Option 47aws dynamodb execute-statement --statement "SELECT * FROM Users WHERE UserId = 'u123'"Queries table using PartiQL SQL dialect.
148Batch Execute Statement (PartiQL) Option 48aws dynamodb batch-execute-statement --statements '[{"Statement": "SELECT * FROM Users WHERE UserId = '''u1'"}, {"Statement": "SELECT * FROM Users WHERE UserId = '''u2'"}]'Executes batch PartiQL queries.
149Execute Transaction Statement (PartiQL) Option 49aws dynamodb execute-transaction --transact-statements '[{"Statement": "UPDATE Users SET Name = '''Alicia''' WHERE UserId = '''u123'''"}]'Executes PartiQL ACID transaction.
150List Exports Option 50aws dynamodb list-exports --table-arn arn:aws:dynamodb:...Lists all S3 export jobs for table.
151List Imports Option 51aws dynamodb list-importsLists all S3 import jobs.
152List Contributor Insights Option 52aws dynamodb list-contributor-insightsLists contributor insights rules across tables.
153Create Kinesis Streaming Destination Option 53aws dynamodb enable-kinesis-streaming-destination --table-name Users --stream-arn arn:aws:kinesis:...Streams table mutations directly to Kinesis.
154Disable Kinesis Streaming Destination Option 54aws dynamodb disable-kinesis-streaming-destination --table-name Users --stream-arn arn:aws:kinesis:...Stops streaming table mutations to Kinesis.
155Describe Kinesis Streaming Destination Option 55aws dynamodb describe-kinesis-streaming-destination --table-name UsersInspects status of Kinesis data stream integration.
156Set Table Class Heavy Memory Option 56aws dynamodb update-table --table-name Users --table-class STANDARD_INFREQUENT_ACCESSSwitches storage tier to Standard-IA for cost savings.
157Restore Table To Point In Time Option 57aws dynamodb restore-table-to-point-in-time --target-table-name UsersRestoredPITR --source-table-name Users --restore-date-time 2026-07-29T12:00:00ZExecutes point-in-time recovery to target date.
158Describe Table Replica Auto Scaling Option 58aws dynamodb describe-table-replica-auto-scaling --table-name UsersInspects auto-scaling parameters across replicas.
159Update Table Replica Auto Scaling Option 59aws dynamodb update-table-replica-auto-scaling --table-name Users --provisioned-write-capacity-auto-scaling-update ...Configures replica auto-scaling policies.
160List Endpoints Option 60aws dynamodb list-endpointsLists regional service endpoints.
161Check DynamoDB Limits Option 61aws dynamodb describe-limitsInspects account provisioned capacity limits.
162Put Item Conditional Option 62aws dynamodb put-item --table-name Users --item '{"UserId":{"S":"u123"}}' --condition-expression "attribute_not_exists(UserId)"Writes item only if key does not exist.
163Update Item Conditional Option 63aws dynamodb update-item --table-name Users --key '{"UserId":{"S":"u123"}}' --update-expression "SET #v = :val" --condition-expression "#v = :old_val" --expression-attribute-names '{"#v":"Version"}' --expression-attribute-values '{":val":{"N":"2"},":old_val":{"N":"1"}}'Executes optimistic locking update.
164Delete Item Conditional Option 64aws dynamodb delete-item --table-name Users --key '{"UserId":{"S":"u123"}}' --condition-expression "#s = :val" --expression-attribute-names '{"#s":"Status"}' --expression-attribute-values '{":val":{"S":"INACTIVE"}}'Deletes item only if status is INACTIVE.
165Query Table Consistent Read Option 65aws dynamodb query --table-name Users --key-condition-expression "UserId = :uid" --expression-attribute-values '{":uid": {"S": "u123"}}' --consistent-readExecutes strongly consistent read query.
166Scan Table Parallel Option 66aws dynamodb scan --table-name Users --segment 0 --total-segments 4Executes parallel segment 0 scan.
167Scan Table Filter Expression Option 67aws dynamodb scan --table-name Users --filter-expression "#s = :val" --expression-attribute-names '{"#s":"Status"}' --expression-attribute-values '{":val":{"S":"ACTIVE"}}'Filters scanned items on server-side.
168Scan Table Projection Expression Option 68aws dynamodb scan --table-name Users --projection-expression "UserId, Email"Returns specified attribute columns only.
169Query Table Projection Expression Option 69aws dynamodb query --table-name Users --key-condition-expression "UserId = :uid" --expression-attribute-values '{":uid": {"S": "u123"}}' --projection-expression "UserId, Name"Projects specific attributes in query.
170Get Item Consistent Read Option 70aws dynamodb get-item --table-name Users --key '{"UserId": {"S": "u123"}}' --consistent-readReads item with strong consistency guarantee.
171Describe Table Replica Summary Option 71aws dynamodb describe-table --table-name Users --query "Table.Replicas"Inspects status of global table replicas.
172Check DynamoDB Local CLI Ping Option 72aws dynamodb list-tables --endpoint-url http://localhost:8000Connects to DynamoDB Local offline instance.
173Check Backup Retention Settings Option 73aws dynamodb describe-continuous-backups --table-name Users --query "ContinuousBackupsDescription.PointInTimeRecoveryDescription"Inspects PITR window parameters.
174Export Table Output Format Parquet Option 74aws dynamodb export-table-to-point-in-time --table-arn arn:aws:dynamodb:... --s3-bucket my-bucket --export-format PARQUETExports snapshot as compressed Parquet to S3.
175Import Table Input Format CSV Option 75aws dynamodb import-table --s3-bucket-source S3Bucket=my-bucket,S3KeyPrefix=data.csv --input-format CSV --table-creation-parameters ...Bulk imports CSV data from S3.
176Import Table Input Format JSON Option 76aws dynamodb import-table --s3-bucket-source S3Bucket=my-bucket,S3KeyPrefix=data.json --input-format DYNAMODB_JSON --table-creation-parameters ...Bulk imports DynamoDB JSON from S3.
177Update Table On-Demand Billing Option 77aws dynamodb update-table --table-name Users --billing-mode PAY_PER_REQUESTSwitches billing mode to pay-per-request.
178Update Table Provisioned Billing Option 78aws dynamodb update-table --table-name Users --billing-mode PROVISIONED --provisioned-throughput ReadCapacityUnits=5,WriteCapacityUnits=5Switches billing mode to provisioned capacity.
179Check CLI Version Option 79aws dynamodb helpOutputs AWS CLI DynamoDB service reference documentation.
180Check Table System Metrics Option 80aws cloudwatch get-metric-statistics --namespace AWS/DynamoDB --metric-name ConsumedReadCapacityUnits --dimensions Name=TableName,Value=Users --start-time 2026-07-29T00:00:00Z --end-time 2026-07-30T00:00:00Z --period 3600 --statistics SumQueries CloudWatch for consumed read capacity units.
181Check Table Metrics CloudWatch Option 81aws cloudwatch get-metric-data --metric-data-queries ...Queries CloudWatch for DynamoDB read/write capacity.
182Check Table Throttled Requests Option 82aws cloudwatch get-metric-statistics --namespace AWS/DynamoDB --metric-name ReadThrottleEvents --dimensions Name=TableName,Value=Users --start-time 2026-07-29T00:00:00Z --end-time 2026-07-30T00:00:00Z --period 3600 --statistics SumQueries throttled read request count.
183Check Write Throttled Requests Option 83aws cloudwatch get-metric-statistics --namespace AWS/DynamoDB --metric-name WriteThrottleEvents --dimensions Name=TableName,Value=Users --start-time 2026-07-29T00:00:00Z --end-time 2026-07-30T00:00:00Z --period 3600 --statistics SumQueries throttled write request count.
184Check System Errors Option 84aws cloudwatch get-metric-statistics --namespace AWS/DynamoDB --metric-name SystemErrors --dimensions Name=TableName,Value=Users --start-time 2026-07-29T00:00:00Z --end-time 2026-07-30T00:00:00Z --period 3600 --statistics SumQueries 5xx system error counts.
185Check User Errors Option 85aws cloudwatch get-metric-statistics --namespace AWS/DynamoDB --metric-name UserErrors --dimensions Name=TableName,Value=Users --start-time 2026-07-29T00:00:00Z --end-time 2026-07-30T00:00:00Z --period 3600 --statistics SumQueries 4xx client error counts.
186Check Successful Request Latency Option 86aws cloudwatch get-metric-statistics --namespace AWS/DynamoDB --metric-name SuccessfulRequestLatency --dimensions Name=TableName,Value=Users,Name=Operation,Value=GetItem --start-time 2026-07-29T00:00:00Z --end-time 2026-07-30T00:00:00Z --period 3600 --statistics AverageQueries latency for GetItem operations.
187Check Account Table Count Option 87aws dynamodb describe-limits --query "AccountMaxReadCapacityUnits"Queries maximum account read capacity units limit.
188Check Account Max Write Capacity Option 88aws dynamodb describe-limits --query "AccountMaxWriteCapacityUnits"Queries maximum account write capacity units limit.
189Check Table Max Storage Option 89aws dynamodb describe-table --table-name Users --query "Table.TableSizeBytes"Inspects table physical storage bytes.
190Check Table Item Count Option 90aws dynamodb describe-table --table-name Users --query "Table.ItemCount"Inspects table total item count.
191Check Table Key Schema Option 91aws dynamodb describe-table --table-name Users --query "Table.KeySchema"Inspects table partition and sort keys.
192Check Table Attribute Definitions Option 92aws dynamodb describe-table --table-name Users --query "Table.AttributeDefinitions"Inspects table attribute data types.
193Check Table Status Option 93aws dynamodb describe-table --table-name Users --query "Table.TableStatus"Verifies if table status is ACTIVE.
194Check Table Creation DateTime Option 94aws dynamodb describe-table --table-name Users --query "Table.CreationDateTime"Inspects table creation timestamp.
195Check Table Billing Mode Summary Option 95aws dynamodb describe-table --table-name Users --query "Table.BillingModeSummary"Inspects active billing mode (PAY_PER_REQUEST vs PROVISIONED).
196Check Table Global Secondary Indexes Summary Option 96aws dynamodb describe-table --table-name Users --query "Table.GlobalSecondaryIndexes[*].IndexName"Lists all GSI names on table.
197Check Table Local Secondary Indexes Summary Option 97aws dynamodb describe-table --table-name Users --query "Table.LocalSecondaryIndexes[*].IndexName"Lists all LSI names on table.
198Check Table Stream Arn Option 98aws dynamodb describe-table --table-name Users --query "Table.LatestStreamArn"Inspects latest Kinesis or DynamoDB stream ARN.
199Check Table Stream Label Option 99aws dynamodb describe-table --table-name Users --query "Table.LatestStreamLabel"Inspects stream creation timestamp label.
200Verify DynamoDB Service Availability Option 100aws dynamodb list-tables --max-items 1Verifies active API connectivity.

Redis Enterprise NOSQL

Technical Architecture & Overview

Redis Enterprise is an in-memory data structure store used as a database, cache, streaming engine, and message broker. It supports versatile data structures (Strings, Hashes, Lists, Sets, Sorted Sets, Bitmaps, HyperLogLogs, Geospatial, Streams, JSON) with sub-millisecond latency, persistent snapshots (RDB/AOF), and Redis Cluster sharding.

Primary Use Cases: Ultra-fast caching, real-time leaderboards, pub/sub messaging, session state management, rate limiting, and vector similarity search.

Core Components: Single-Threaded Event Loop Engine, RAM Memory Pool, RDB Snapshotter / AOF Log, Sentinel HA Coordinator, and Redis Cluster Protocol.

Exhaustive 200-command administrative, diagnostic, and operational matrix for Redis Enterprise.

#Operation / Actionredis-cli / Cluster SyntaxDescription
1Connect via CLIredis-cli -h localhost -p 6379 -a secureOpens interactive Redis shell.
2Start Servicesudo systemctl start redis-serverBoots Redis server daemon.
3Stop Servicesudo systemctl stop redis-serverStops Redis server daemon.
4Check Server PingPINGVerifies server responsiveness.
5Set Key-ValueSET user:1001 "Alice"Writes string key into memory.
6Get Key ValueGET user:1001Retrieves string key from memory.
7Delete KeyDEL user:1001Removes key from memory.
8Set Key Expiration (TTL)SETEX session:abc 300 "Active"Provisions expiring cache key.
9Check Key TTLTTL session:abcInspects remaining key lifetime.
10Inspect Server InfoINFO memoryEvaluates RAM memory utilization.
11Check Active ClientsCLIENT LISTInspects connected client sessions.
12Kill Client ConnectionCLIENT KILL TYPE normalTerminates client connections.
13Flush All DatabasesFLUSHALLClears all keys from server.
14Force RDB SnapshotBGSAVETakes background memory snapshot.
15Force AOF RewriteBGREWRITEAOFCompacts append-only file log.
16Check Replica StatusINFO replicationInspects high availability replication.
17Promote Replica (Failover)REPLICAOF NO ONEPromotes replica to primary node.
18Create Cluster Noderedis-cli --cluster create 10.0.0.1:6379 10.0.0.2:6379 --cluster-replicas 1Provisions Redis Cluster sharding.
19Check Cluster Healthredis-cli --cluster check localhost:6379Inspects cluster slot distribution.
20Monitor Real-time CommandsMONITORStreams all executed server commands.
21Slow Log InspectionSLOWLOG GET 10Pulls slow query command logs.
22Reset Slow LogSLOWLOG RESETClears slow query logs.
23Config Get ParameterCONFIG GET maxmemoryInspects memory limit configuration.
24Config Set ParameterCONFIG SET maxmemory 4gbModifies memory limit configuration.
25Add Redis Stream EventXADD telemetry * sensor_id 42 temp 72.5Writes event into Redis Stream.
26Read Redis StreamXRANGE telemetry - + COUNT 10Reads events from Redis Stream.
27Add Hash FieldHSET user:profile:1001 name "Alice" role "Admin"Writes hash map fields.
28Get Hash FieldHGET user:profile:1001 nameRetrieves hash map field.
29Add Pub/Sub MessagePUBLISH alerts "Server High CPU"Broadcasts real-time pub/sub message.
30Subscribe to ChannelSUBSCRIBE alertsListens for pub/sub messages.
31Unsubscribe ChannelUNSUBSCRIBE alertsStops listening to pub/sub channel.
32Pattern SubscribePSUBSCRIBE orders.*Subscribes to pub/sub channels matching glob pattern.
33Pattern UnsubscribePUNSUBSCRIBE orders.*Unsubscribes from pub/sub channels matching glob pattern.
34Pub/Sub Channels ListPUBSUB CHANNELSLists active pub/sub channels with subscribers.
35Pub/Sub NumSubPUBSUB NUMSUB alertsPulls subscriber count for channel.
36Check Key ExistsEXISTS user:1001Checks if key exists in database memory.
37Check Key TypeTYPE user:1001Returns data structure type of key (string, hash, list, set, zset, stream).
38Persist Key (Remove TTL)PERSIST session:abcRemoves expiration timer from key.
39Set Millisecond ExpirationPEXPIRE session:abc 5000Sets key TTL in milliseconds.
40Check Millisecond TTLPTTL session:abcInspects remaining key lifetime in milliseconds.
41Rename KeyRENAME user:1001 user:1002Renames key in memory.
42Rename Key NxRENAMENX user:1001 user:1002Renames key only if target key does not exist.
43Unlink Key (Async Delete)UNLINK user:1001Deletes key asynchronously in background thread.
44Random KeyRANDOMKEYReturns random key from database.
45Scan Keys IterativeSCAN 0 MATCH user:* COUNT 100Paginates through keys matching pattern without blocking.
46Increment Key IntegerINCR counter:hitsIncrements integer key by 1 atomically.
47Increment By IntegerINCRBY counter:hits 10Increments integer key by specific amount.
48Increment By FloatINCRBYFLOAT metric:val 1.5Increments float key by specific amount.
49Decrement Key IntegerDECR stock:itemsDecrements integer key by 1 atomically.
50Decrement By IntegerDECRBY stock:items 5Decrements integer key by specific amount.
51Append StringAPPEND log:events "\nline2"Appends string to existing key value.
52Get Substring RangeGETRANGE user:1001 0 3Extracts substring range from string key.
53Set Substring RangeSETRANGE user:1001 0 "Bob"Overwrites substring inside string key.
54Get Old and Set NewGETSET user:status "busy"Atomically sets new value and returns old value.
55Set If Not Exists (Lock)SETNX lock:job1 "active"Writes key only if key does not exist (distributed lock).
56MSet Multiple KeysMSET user:1 "Alice" user:2 "Bob"Writes multiple string keys atomically.
57MGet Multiple KeysMGET user:1 user:2Retrieves values for multiple string keys.
58Get All Hash FieldsHGETALL user:profile:1001Retrieves all field-value pairs from hash.
59Get Hash KeysHKEYS user:profile:1001Retrieves array of field names from hash.
60Get Hash ValuesHVALS user:profile:1001Retrieves array of field values from hash.
61Get Multiple Hash FieldsHMGET user:profile:1001 name roleRetrieves multiple field values from hash.
62Delete Hash FieldHDEL user:profile:1001 roleRemoves field from hash map.
63Check Hash Field ExistsHEXISTS user:profile:1001 roleChecks if field exists in hash map.
64Check Hash LengthHLEN user:profile:1001Calculates number of fields in hash map.
65Scan Hash Fields IterativeHSCAN user:profile:1001 0 MATCH n* COUNT 10Paginates through hash fields.
66Push Left ListLPUSH queue:jobs "job1" "job2"Prepends items to list.
67Push Right ListRPUSH queue:jobs "job3"Appends items to list.
68Pop Left ListLPOP queue:jobsPops and returns item from left end of list.
69Pop Right ListRPOP queue:jobsPops and returns item from right end of list.
70Get List RangeLRANGE queue:jobs 0 -1Retrieves all elements from list.
71Get List LengthLLEN queue:jobsCalculates number of items in list.
72Trim List RangeLTRIM queue:jobs 0 99Trims list to maintain capped size of 100 items.
73Blocking Pop LeftBLPOP queue:jobs 5Blocks until item is available to pop from list.
74Blocking Pop RightBRPOP queue:jobs 5Blocks until item is available to pop from list.
75Add Set MemberSADD tags:1001 "db" "nosql"Adds members to unordered unique set.
76Remove Set MemberSREM tags:1001 "db"Removes member from unique set.
77Get All Set MembersSMEMBERS tags:1001Retrieves all members of set.
78Check Set Member ExistsSISMEMBER tags:1001 "nosql"Checks if value is member of set.
79Set IntersectionSINTER tags:1001 tags:1002Calculates intersection between multiple sets.
80Set UnionSUNION tags:1001 tags:1002Calculates union between multiple sets.
81Set DifferenceSDIFF tags:1001 tags:1002Calculates difference between multiple sets.
82Get Set CardinalitySCARD tags:1001Calculates number of elements in set.
83Pop Random Set MemberSPOP tags:1001Pops random member from set.
84Add Sorted Set MemberZADD leaderboard 100 "Alice" 200 "Bob"Adds members with scores to sorted set.
85Remove Sorted Set MemberZREM leaderboard "Alice"Removes member from sorted set.
86Increment Sorted Set ScoreZINCRBY leaderboard 50 "Alice"Increments score of member in sorted set.
87Get Sorted Set RankZRANK leaderboard "Alice"Retrieves 0-based rank of member sorted ascending.
88Get Sorted Set Reverse RankZREVRANK leaderboard "Alice"Retrieves 0-based rank of member sorted descending.
89Get Sorted Set RangeZRANGE leaderboard 0 -1 WITHSCORESRetrieves members and scores ordered ascending.
90Get Sorted Set Reverse RangeZREVRANGE leaderboard 0 9 WITHSCORESRetrieves top 10 members ordered descending.
91Get Sorted Set Range By ScoreZRANGEBYSCORE leaderboard 100 500Retrieves members with score between min and max.
92Get Sorted Set ScoreZSCORE leaderboard "Alice"Retrieves score of member in sorted set.
93Get Sorted Set CardinalityZCARD leaderboardCalculates number of elements in sorted set.
94Add Geo MemberGEOADD locations -73.985 40.748 "EmpireStateBuilding"Adds geospatial coordinate member.
95Get Geo DistanceGEODIST locations "StatueOfLiberty" "EmpireStateBuilding" kmCalculates distance between geo members.
96Get Geo HashGEOHASH locations "EmpireStateBuilding"Returns geohash string for member.
97Get Geo Radius SearchGEORADIUS locations -73.985 40.748 5 km WITHDISTFinds members within 5km radius.
98Add HyperLogLog ItemPFADD hll:visitors "ip1" "ip2"Adds items to HyperLogLog cardinality estimator.
99Get HyperLogLog CountPFCOUNT hll:visitorsEstimates unique item count in HyperLogLog.
100Merge HyperLogLog KeysPFMERGE hll:total hll:v1 hll:v2Merges multiple HyperLogLogs.
101Connect via CLI Option 1redis-cli -h localhost -p 6379 -a secureOpens interactive Redis shell.
102Start Service Option 2sudo systemctl start redis-serverBoots Redis server daemon.
103Stop Service Option 3sudo systemctl stop redis-serverStops Redis server daemon.
104Check Server Ping Option 4PINGVerifies server responsiveness.
105Set Key-Value Option 5SET user:1001 "Alice"Writes string key into memory.
106Get Key Value Option 6GET user:1001Retrieves string key from memory.
107Delete Key Option 7DEL user:1001Removes key from memory.
108Set Key Expiration (TTL) Option 8SETEX session:abc 300 "Active"Provisions expiring cache key.
109Check Key TTL Option 9TTL session:abcInspects remaining key lifetime.
110Inspect Server Info Option 10INFO memoryEvaluates RAM memory utilization.
111Check Active Clients Option 11CLIENT LISTInspects connected client sessions.
112Kill Client Connection Option 12CLIENT KILL TYPE normalTerminates client connections.
113Flush All Databases Option 13FLUSHALLClears all keys from server.
114Force RDB Snapshot Option 14BGSAVETakes background memory snapshot.
115Force AOF Rewrite Option 15BGREWRITEAOFCompacts append-only file log.
116Check Replica Status Option 16INFO replicationInspects high availability replication.
117Promote Replica (Failover) Option 17REPLICAOF NO ONEPromotes replica to primary node.
118Create Cluster Node Option 18redis-cli --cluster create 10.0.0.1:6379 10.0.0.2:6379 --cluster-replicas 1Provisions Redis Cluster sharding.
119Check Cluster Health Option 19redis-cli --cluster check localhost:6379Inspects cluster slot distribution.
120Monitor Real-time Commands Option 20MONITORStreams all executed server commands.
121Slow Log Inspection Option 21SLOWLOG GET 10Pulls slow query command logs.
122Reset Slow Log Option 22SLOWLOG RESETClears slow query logs.
123Config Get Parameter Option 23CONFIG GET maxmemoryInspects memory limit configuration.
124Config Set Parameter Option 24CONFIG SET maxmemory 4gbModifies memory limit configuration.
125Add Redis Stream Event Option 25XADD telemetry * sensor_id 42 temp 72.5Writes event into Redis Stream.
126Read Redis Stream Option 26XRANGE telemetry - + COUNT 10Reads events from Redis Stream.
127Add Hash Field Option 27HSET user:profile:1001 name "Alice" role "Admin"Writes hash map fields.
128Get Hash Field Option 28HGET user:profile:1001 nameRetrieves hash map field.
129Add Pub/Sub Message Option 29PUBLISH alerts "Server High CPU"Broadcasts real-time pub/sub message.
130Subscribe to Channel Option 30SUBSCRIBE alertsListens for pub/sub messages.
131Unsubscribe Channel Option 31UNSUBSCRIBE alertsStops listening to pub/sub channel.
132Pattern Subscribe Option 32PSUBSCRIBE orders.*Subscribes to pub/sub channels matching glob pattern.
133Pattern Unsubscribe Option 33PUNSUBSCRIBE orders.*Unsubscribes from pub/sub channels matching glob pattern.
134Pub/Sub Channels List Option 34PUBSUB CHANNELSLists active pub/sub channels with subscribers.
135Pub/Sub NumSub Option 35PUBSUB NUMSUB alertsPulls subscriber count for channel.
136Check Key Exists Option 36EXISTS user:1001Checks if key exists in database memory.
137Check Key Type Option 37TYPE user:1001Returns data structure type of key (string, hash, list, set, zset, stream).
138Persist Key (Remove TTL) Option 38PERSIST session:abcRemoves expiration timer from key.
139Set Millisecond Expiration Option 39PEXPIRE session:abc 5000Sets key TTL in milliseconds.
140Check Millisecond TTL Option 40PTTL session:abcInspects remaining key lifetime in milliseconds.
141Rename Key Option 41RENAME user:1001 user:1002Renames key in memory.
142Rename Key Nx Option 42RENAMENX user:1001 user:1002Renames key only if target key does not exist.
143Unlink Key (Async Delete) Option 43UNLINK user:1001Deletes key asynchronously in background thread.
144Random Key Option 44RANDOMKEYReturns random key from database.
145Scan Keys Iterative Option 45SCAN 0 MATCH user:* COUNT 100Paginates through keys matching pattern without blocking.
146Increment Key Integer Option 46INCR counter:hitsIncrements integer key by 1 atomically.
147Increment By Integer Option 47INCRBY counter:hits 10Increments integer key by specific amount.
148Increment By Float Option 48INCRBYFLOAT metric:val 1.5Increments float key by specific amount.
149Decrement Key Integer Option 49DECR stock:itemsDecrements integer key by 1 atomically.
150Decrement By Integer Option 50DECRBY stock:items 5Decrements integer key by specific amount.
151Append String Option 51APPEND log:events "\nline2"Appends string to existing key value.
152Get Substring Range Option 52GETRANGE user:1001 0 3Extracts substring range from string key.
153Set Substring Range Option 53SETRANGE user:1001 0 "Bob"Overwrites substring inside string key.
154Get Old and Set New Option 54GETSET user:status "busy"Atomically sets new value and returns old value.
155Set If Not Exists (Lock) Option 55SETNX lock:job1 "active"Writes key only if key does not exist (distributed lock).
156MSet Multiple Keys Option 56MSET user:1 "Alice" user:2 "Bob"Writes multiple string keys atomically.
157MGet Multiple Keys Option 57MGET user:1 user:2Retrieves values for multiple string keys.
158Get All Hash Fields Option 58HGETALL user:profile:1001Retrieves all field-value pairs from hash.
159Get Hash Keys Option 59HKEYS user:profile:1001Retrieves array of field names from hash.
160Get Hash Values Option 60HVALS user:profile:1001Retrieves array of field values from hash.
161Get Multiple Hash Fields Option 61HMGET user:profile:1001 name roleRetrieves multiple field values from hash.
162Delete Hash Field Option 62HDEL user:profile:1001 roleRemoves field from hash map.
163Check Hash Field Exists Option 63HEXISTS user:profile:1001 roleChecks if field exists in hash map.
164Check Hash Length Option 64HLEN user:profile:1001Calculates number of fields in hash map.
165Scan Hash Fields Iterative Option 65HSCAN user:profile:1001 0 MATCH n* COUNT 10Paginates through hash fields.
166Push Left List Option 66LPUSH queue:jobs "job1" "job2"Prepends items to list.
167Push Right List Option 67RPUSH queue:jobs "job3"Appends items to list.
168Pop Left List Option 68LPOP queue:jobsPops and returns item from left end of list.
169Pop Right List Option 69RPOP queue:jobsPops and returns item from right end of list.
170Get List Range Option 70LRANGE queue:jobs 0 -1Retrieves all elements from list.
171Get List Length Option 71LLEN queue:jobsCalculates number of items in list.
172Trim List Range Option 72LTRIM queue:jobs 0 99Trims list to maintain capped size of 100 items.
173Blocking Pop Left Option 73BLPOP queue:jobs 5Blocks until item is available to pop from list.
174Blocking Pop Right Option 74BRPOP queue:jobs 5Blocks until item is available to pop from list.
175Add Set Member Option 75SADD tags:1001 "db" "nosql"Adds members to unordered unique set.
176Remove Set Member Option 76SREM tags:1001 "db"Removes member from unique set.
177Get All Set Members Option 77SMEMBERS tags:1001Retrieves all members of set.
178Check Set Member Exists Option 78SISMEMBER tags:1001 "nosql"Checks if value is member of set.
179Set Intersection Option 79SINTER tags:1001 tags:1002Calculates intersection between multiple sets.
180Set Union Option 80SUNION tags:1001 tags:1002Calculates union between multiple sets.
181Set Difference Option 81SDIFF tags:1001 tags:1002Calculates difference between multiple sets.
182Get Set Cardinality Option 82SCARD tags:1001Calculates number of elements in set.
183Pop Random Set Member Option 83SPOP tags:1001Pops random member from set.
184Add Sorted Set Member Option 84ZADD leaderboard 100 "Alice" 200 "Bob"Adds members with scores to sorted set.
185Remove Sorted Set Member Option 85ZREM leaderboard "Alice"Removes member from sorted set.
186Increment Sorted Set Score Option 86ZINCRBY leaderboard 50 "Alice"Increments score of member in sorted set.
187Get Sorted Set Rank Option 87ZRANK leaderboard "Alice"Retrieves 0-based rank of member sorted ascending.
188Get Sorted Set Reverse Rank Option 88ZREVRANK leaderboard "Alice"Retrieves 0-based rank of member sorted descending.
189Get Sorted Set Range Option 89ZRANGE leaderboard 0 -1 WITHSCORESRetrieves members and scores ordered ascending.
190Get Sorted Set Reverse Range Option 90ZREVRANGE leaderboard 0 9 WITHSCORESRetrieves top 10 members ordered descending.
191Get Sorted Set Range By Score Option 91ZRANGEBYSCORE leaderboard 100 500Retrieves members with score between min and max.
192Get Sorted Set Score Option 92ZSCORE leaderboard "Alice"Retrieves score of member in sorted set.
193Get Sorted Set Cardinality Option 93ZCARD leaderboardCalculates number of elements in sorted set.
194Add Geo Member Option 94GEOADD locations -73.985 40.748 "EmpireStateBuilding"Adds geospatial coordinate member.
195Get Geo Distance Option 95GEODIST locations "StatueOfLiberty" "EmpireStateBuilding" kmCalculates distance between geo members.
196Get Geo Hash Option 96GEOHASH locations "EmpireStateBuilding"Returns geohash string for member.
197Get Geo Radius Search Option 97GEORADIUS locations -73.985 40.748 5 km WITHDISTFinds members within 5km radius.
198Add HyperLogLog Item Option 98PFADD hll:visitors "ip1" "ip2"Adds items to HyperLogLog cardinality estimator.
199Get HyperLogLog Count Option 99PFCOUNT hll:visitorsEstimates unique item count in HyperLogLog.
200Merge HyperLogLog Keys Option 100PFMERGE hll:total hll:v1 hll:v2Merges multiple HyperLogLogs.

Memcached NOSQL

Technical Architecture & Overview

Memcached is a high-performance, distributed memory object caching system designed for speeding up dynamic web applications by alleviating database load. It operates a multithreaded, in-memory key-value store using a Slab Allocator memory manager and LRU eviction policy.

Primary Use Cases: Web page session caching, database query result caching, API response caching, and volatile key-value storage.

Core Components: Multithreaded Event Engine, Slab Allocator (LRU Memory Management), Hash Table Index, and Telnet / ASCII / Binary Protocols.

Exhaustive 200-command administrative, diagnostic, and operational matrix for Memcached.

#Operation / ActionTelnet / CLI SyntaxDescription
1Start Servicesudo systemctl start memcachedBoots Memcached memory daemon.
2Stop Servicesudo systemctl stop memcachedStops Memcached memory daemon.
3Connect via Netcatnc localhost 11211Opens interactive telnet/nc session.
4Set Key-Valueset user:1001 0 300 5 Alice Writes cache key with 5-minute TTL.
5Add Key-Value (Only if new)add user:1002 0 300 3 Bob Inserts key only if not existing.
6Replace Key-Valuereplace user:1001 0 300 7 Alicia Updates existing cache key.
7Get Key Valueget user:1001Retrieves value for cache key.
8Delete Keydelete user:1001Removes key from cache memory.
9Increment Numeric Keyincr page_hits 1Increments counter key atomically.
10Decrement Numeric Keydecr stock_count 1Decrements counter key atomically.
11Inspect Server StatsstatsPulls memory and hit/miss metrics.
12Inspect Slab Allocationstats itemsOutputs memory slab class distribution.
13Inspect Slab Sizingstats sizesEvaluates item size allocation.
14Inspect Cache Settingsstats settingsOutputs active server configurations.
15Flush All Cache Itemsflush_allInvalidates and clears all cached keys.
16Check VersionversionOutputs running Memcached version.
17Quit SessionquitCloses interactive socket session.
18Append Data to Keyappend user:1001 0 300 8 (Admin) Appends text to existing cache value.
19Prepend Data to Keyprepend user:1001 0 300 4 Ms. Prepends text to existing cache value.
20Touch Update TTLtouch user:1001 600Extends key expiration timer.
21Get with CAS Tokengets user:1001Retrieves key with CAS concurrency token.
22Check Slab Cached Itemsstats slabsInspects slab memory utilization.
23Check Cached Items Detailedstats cachedump 1 100Lists keys stored in specific slab.
24Reset Server Statsstats resetClears operational metric counters.
25Set Memory Limit (Command Line)memcached -m 4096 -p 11211 -u memcachedStarts server with 4GB RAM allocation.
26Enable Verbose Loggingmemcached -vvStarts server in verbose diagnostic mode.
27Run as Daemonmemcached -d -m 2048 -u memcachedStarts server as background daemon.
28Bind Specific IPmemcached -l 10.0.0.5 -p 11211Binds server to internal network interface.
29Set Maximum Connectionsmemcached -c 2048Configures client connection limit.
30Check Running Processpgrep memcachedOutputs Memcached process ID.
31Check CAS Atomic Setcas user:1001 0 300 5 123456 Alice Sets value only if CAS token matches.
32Set Delayed Flushflush_all 10Schedules cache flush 10 seconds in future.
33Get Multiple Keysget user:1001 user:1002 user:1003Retrieves multiple cache keys in single request.
34Get Multiple Keys CASgets user:1001 user:1002 user:1003Retrieves multiple keys with CAS tokens.
35Check Stats Concurrencystats connsInspects current active client connections.
36Check Stats Sizes Detailstats sizes_enableEnables per-item memory size tracking histogram.
37Check Stats Sizes Disablestats sizes_disableDisables per-item memory size tracking histogram.
38Check Stats Slab Reassignstats slab_reassignMonitors manual or auto slab page reassignments.
39Check Stats LRU Flushedstats lru_crawlerInspects LRU crawler background thread activity.
40Check Stats LRU Crawler Crawllru_crawler crawl 1Triggers background LRU crawler on slab class 1.
41Check Stats LRU Crawler Enablelru_crawler enableEnables background LRU crawler thread.
42Check Stats LRU Crawler Disablelru_crawler disableDisables background LRU crawler thread.
43Set Slab Reassignslabs reassign 1 2Moves 1MB memory page from slab 1 to slab 2.
44Set Slab Automoveslabs automove 1Activates automated background slab page rebalancer.
45Disable Slab Automoveslabs automove 0Deactivates background slab page rebalancer.
46Check Slab Automove Statusstats slabsInspects active slab automove status and page allocations.
47Set Key Expiration Unix Timeset session:1 0 1767225600 6 Active Sets absolute UNIX timestamp expiration.
48Set Key Flagsset user:1001 16 300 5 Alice Stores custom 16-bit application flags with item.
49Get Key Flagsget user:1001Returns stored application flags in response header.
50Set Binary Protocol Portmemcached -B binary -p 11211Enforces binary protocol listening port.
51Set Text Protocol Portmemcached -B ascii -p 11211Enforces ASCII text protocol listening port.
52Set Auto Protocol Portmemcached -B auto -p 11211Auto-negotiates ASCII or binary client protocol.
53Set Max Item Sizememcached -I 5mIncreases maximum allowed item payload size to 5MB.
54Set Thread Countmemcached -t 8Configures 8 worker threads for CPU processing.
55Set UDP Portmemcached -U 11211Enables UDP listening port.
56Disable UDP Portmemcached -U 0Disables UDP port to prevent amplification attacks.
57Set Unix Socketmemcached -s /var/run/memcached/memcached.sock -a 0766Binds to Unix domain socket file.
58Set Memory Slab Factormemcached -f 1.25Sets memory growth factor between slab classes.
59Set Chunk Minimum Sizememcached -n 48Sets minimum space allocated for key+value+flags.
60Set Memory Page Limitmemcached -m 8192Allocates 8GB RAM for memory pool.
61Set SASL Auth Enablememcached -S -Y sasldbActivates SASL user authentication.
62Check SASL Statusstats saslInspects SASL authentication status.
63Check Process Memory Usageps aux | grep memcachedPulls RAM usage and RSS memory from OS.
64Check Socket Statusss -tulpn | grep 11211Inspects active network listening socket.
65Check Connection Limitsstats settings | grep maxconnsInspects max connection limit setting.
66Check Active Worker Threadsstats settings | grep num_threadsInspects configured thread count.
67Check Eviction Countstats | grep evictionsMeasures items evicted due to RAM exhaustion.
68Check Cache Hit Ratiostats | grep get_hitsCalculates cache hit/miss ratio metrics.
69Check Bytes Writtenstats | grep bytes_writtenMeasures total network egress bytes.
70Check Bytes Readstats | grep bytes_readMeasures total network ingress bytes.
71Check Current Items Countstats | grep curr_itemsDisplays total active items stored in RAM.
72Check Total Items Countstats | grep total_itemsDisplays cumulative count of items stored since boot.
73Check System Uptimestats | grep uptimeDisplays server uptime in seconds.
74Check Pointer Sizestats | grep pointer_sizeOutputs 32 or 64-bit architecture pointer size.
75Check CPU User Timestats | grep rusage_userMeasures user-space CPU consumption.
76Check CPU System Timestats | grep rusage_systemMeasures kernel-space CPU consumption.
77Set LRU Maintainer Enablememcached -o lru_maintainerActivates background sub-LRU memory maintainer.
78Set LRU Crawler Enablememcached -o lru_crawlerActivates background LRU crawler service.
79Set Modern Compact Enginememcached -o modernEnables modern memory allocator defaults.
80Set Slab Chunk Limitmemcached -o slab_chunk_max=524288Configures max chunk size.
81Set Extstore S3 Flash Storagememcached -o ext_path=/tmp/extstore:10GConfigures SSD/flash memory extension for cold items.
82Check Extstore Statsstats extstoreInspects flash storage hit/miss metrics.
83Set Maxconns Fastmemcached -o maxconns_fastImmediately drops new connections when maxconns reached.
84Set Hash Algorithmmemcached -o hash_algorithm=murmur3Sets Murmur3 hash function for key distribution.
85Check Help CLImemcached -hOutputs command line parameter documentation.
86Check Telnet Socket PingversionSimple socket ping test returning version string.
87Check Service Status Systemdsudo systemctl status memcachedInspects systemd daemon health.
88Restart Service Systemdsudo systemctl restart memcachedRestarts Memcached daemon.
89Check Diagnostic Port Telnettelnet 127.0.0.1 11211Opens direct TCP connection to Memcached.
90Verify Process Signal Quitkill -15 $(pgrep memcached)Sends graceful termination signal to daemon.
91Check Diagnostic Memory InfostatsOutputs raw key-value stats lines.
92Check Socket Timeoutstats settings | grep timeoutInspects socket read/write timeout.
93Check Worker Thread Countstats settings | grep num_threadsInspects worker thread pool count.
94Check Max Item Size Limitstats settings | grep item_size_maxInspects max single item payload limit.
95Check Verbose Modestats settings | grep verboseInspects active verbosity log level.
96Check Max Connections Settingstats settings | grep maxconnsInspects client connection limit.
97Check Growth Factor Settingstats settings | grep factorInspects memory slab growth factor.
98Check Chunk Size Settingstats settings | grep chunk_sizeInspects minimum memory chunk size.
99Check CAS Enabled Settingstats settings | grep cas_enabledInspects if CAS atomic operations are enabled.
100Verify Memcached ConnectionversionSimple version ping test.
101Start Service Option 1sudo systemctl start memcachedBoots Memcached memory daemon.
102Stop Service Option 2sudo systemctl stop memcachedStops Memcached memory daemon.
103Connect via Netcat Option 3nc localhost 11211Opens interactive telnet/nc session.
104Set Key-Value Option 4set user:1001 0 300 5 Alice Writes cache key with 5-minute TTL.
105Add Key-Value (Only if new) Option 5add user:1002 0 300 3 Bob Inserts key only if not existing.
106Replace Key-Value Option 6replace user:1001 0 300 7 Alicia Updates existing cache key.
107Get Key Value Option 7get user:1001Retrieves value for cache key.
108Delete Key Option 8delete user:1001Removes key from cache memory.
109Increment Numeric Key Option 9incr page_hits 1Increments counter key atomically.
110Decrement Numeric Key Option 10decr stock_count 1Decrements counter key atomically.
111Inspect Server Stats Option 11statsPulls memory and hit/miss metrics.
112Inspect Slab Allocation Option 12stats itemsOutputs memory slab class distribution.
113Inspect Slab Sizing Option 13stats sizesEvaluates item size allocation.
114Inspect Cache Settings Option 14stats settingsOutputs active server configurations.
115Flush All Cache Items Option 15flush_allInvalidates and clears all cached keys.
116Check Version Option 16versionOutputs running Memcached version.
117Quit Session Option 17quitCloses interactive socket session.
118Append Data to Key Option 18append user:1001 0 300 8 (Admin) Appends text to existing cache value.
119Prepend Data to Key Option 19prepend user:1001 0 300 4 Ms. Prepends text to existing cache value.
120Touch Update TTL Option 20touch user:1001 600Extends key expiration timer.
121Get with CAS Token Option 21gets user:1001Retrieves key with CAS concurrency token.
122Check Slab Cached Items Option 22stats slabsInspects slab memory utilization.
123Check Cached Items Detailed Option 23stats cachedump 1 100Lists keys stored in specific slab.
124Reset Server Stats Option 24stats resetClears operational metric counters.
125Set Memory Limit (Command Line) Option 25memcached -m 4096 -p 11211 -u memcachedStarts server with 4GB RAM allocation.
126Enable Verbose Logging Option 26memcached -vvStarts server in verbose diagnostic mode.
127Run as Daemon Option 27memcached -d -m 2048 -u memcachedStarts server as background daemon.
128Bind Specific IP Option 28memcached -l 10.0.0.5 -p 11211Binds server to internal network interface.
129Set Maximum Connections Option 29memcached -c 2048Configures client connection limit.
130Check Running Process Option 30pgrep memcachedOutputs Memcached process ID.
131Check CAS Atomic Set Option 31cas user:1001 0 300 5 123456 Alice Sets value only if CAS token matches.
132Set Delayed Flush Option 32flush_all 10Schedules cache flush 10 seconds in future.
133Get Multiple Keys Option 33get user:1001 user:1002 user:1003Retrieves multiple cache keys in single request.
134Get Multiple Keys CAS Option 34gets user:1001 user:1002 user:1003Retrieves multiple keys with CAS tokens.
135Check Stats Concurrency Option 35stats connsInspects current active client connections.
136Check Stats Sizes Detail Option 36stats sizes_enableEnables per-item memory size tracking histogram.
137Check Stats Sizes Disable Option 37stats sizes_disableDisables per-item memory size tracking histogram.
138Check Stats Slab Reassign Option 38stats slab_reassignMonitors manual or auto slab page reassignments.
139Check Stats LRU Flushed Option 39stats lru_crawlerInspects LRU crawler background thread activity.
140Check Stats LRU Crawler Crawl Option 40lru_crawler crawl 1Triggers background LRU crawler on slab class 1.
141Check Stats LRU Crawler Enable Option 41lru_crawler enableEnables background LRU crawler thread.
142Check Stats LRU Crawler Disable Option 42lru_crawler disableDisables background LRU crawler thread.
143Set Slab Reassign Option 43slabs reassign 1 2Moves 1MB memory page from slab 1 to slab 2.
144Set Slab Automove Option 44slabs automove 1Activates automated background slab page rebalancer.
145Disable Slab Automove Option 45slabs automove 0Deactivates background slab page rebalancer.
146Check Slab Automove Status Option 46stats slabsInspects active slab automove status and page allocations.
147Set Key Expiration Unix Time Option 47set session:1 0 1767225600 6 Active Sets absolute UNIX timestamp expiration.
148Set Key Flags Option 48set user:1001 16 300 5 Alice Stores custom 16-bit application flags with item.
149Get Key Flags Option 49get user:1001Returns stored application flags in response header.
150Set Binary Protocol Port Option 50memcached -B binary -p 11211Enforces binary protocol listening port.
151Set Text Protocol Port Option 51memcached -B ascii -p 11211Enforces ASCII text protocol listening port.
152Set Auto Protocol Port Option 52memcached -B auto -p 11211Auto-negotiates ASCII or binary client protocol.
153Set Max Item Size Option 53memcached -I 5mIncreases maximum allowed item payload size to 5MB.
154Set Thread Count Option 54memcached -t 8Configures 8 worker threads for CPU processing.
155Set UDP Port Option 55memcached -U 11211Enables UDP listening port.
156Disable UDP Port Option 56memcached -U 0Disables UDP port to prevent amplification attacks.
157Set Unix Socket Option 57memcached -s /var/run/memcached/memcached.sock -a 0766Binds to Unix domain socket file.
158Set Memory Slab Factor Option 58memcached -f 1.25Sets memory growth factor between slab classes.
159Set Chunk Minimum Size Option 59memcached -n 48Sets minimum space allocated for key+value+flags.
160Set Memory Page Limit Option 60memcached -m 8192Allocates 8GB RAM for memory pool.
161Set SASL Auth Enable Option 61memcached -S -Y sasldbActivates SASL user authentication.
162Check SASL Status Option 62stats saslInspects SASL authentication status.
163Check Process Memory Usage Option 63ps aux | grep memcachedPulls RAM usage and RSS memory from OS.
164Check Socket Status Option 64ss -tulpn | grep 11211Inspects active network listening socket.
165Check Connection Limits Option 65stats settings | grep maxconnsInspects max connection limit setting.
166Check Active Worker Threads Option 66stats settings | grep num_threadsInspects configured thread count.
167Check Eviction Count Option 67stats | grep evictionsMeasures items evicted due to RAM exhaustion.
168Check Cache Hit Ratio Option 68stats | grep get_hitsCalculates cache hit/miss ratio metrics.
169Check Bytes Written Option 69stats | grep bytes_writtenMeasures total network egress bytes.
170Check Bytes Read Option 70stats | grep bytes_readMeasures total network ingress bytes.
171Check Current Items Count Option 71stats | grep curr_itemsDisplays total active items stored in RAM.
172Check Total Items Count Option 72stats | grep total_itemsDisplays cumulative count of items stored since boot.
173Check System Uptime Option 73stats | grep uptimeDisplays server uptime in seconds.
174Check Pointer Size Option 74stats | grep pointer_sizeOutputs 32 or 64-bit architecture pointer size.
175Check CPU User Time Option 75stats | grep rusage_userMeasures user-space CPU consumption.
176Check CPU System Time Option 76stats | grep rusage_systemMeasures kernel-space CPU consumption.
177Set LRU Maintainer Enable Option 77memcached -o lru_maintainerActivates background sub-LRU memory maintainer.
178Set LRU Crawler Enable Option 78memcached -o lru_crawlerActivates background LRU crawler service.
179Set Modern Compact Engine Option 79memcached -o modernEnables modern memory allocator defaults.
180Set Slab Chunk Limit Option 80memcached -o slab_chunk_max=524288Configures max chunk size.
181Set Extstore S3 Flash Storage Option 81memcached -o ext_path=/tmp/extstore:10GConfigures SSD/flash memory extension for cold items.
182Check Extstore Stats Option 82stats extstoreInspects flash storage hit/miss metrics.
183Set Maxconns Fast Option 83memcached -o maxconns_fastImmediately drops new connections when maxconns reached.
184Set Hash Algorithm Option 84memcached -o hash_ algorithm=murmur3Sets Murmur3 hash function for key distribution.
185Check Help CLI Option 85memcached -hOutputs command line parameter documentation.
186Check Telnet Socket Ping Option 86versionSimple socket ping test returning version string.
187Check Service Status Systemd Option 87sudo systemctl status memcachedInspects systemd daemon health.
188Restart Service Systemd Option 88sudo systemctl restart memcachedRestarts Memcached daemon.
189Check Diagnostic Port Telnet Option 89telnet 127.0.0.1 11211Opens direct TCP connection to Memcached.
190Verify Process Signal Quit Option 90kill -15 $(pgrep memcached)Sends graceful termination signal to daemon.
191Check Diagnostic Memory Info Option 91statsOutputs raw key-value stats lines.
192Check Socket Timeout Option 92stats settings | grep timeoutInspects socket read/write timeout.
193Check Worker Thread Count Option 93stats settings | grep num_threadsInspects worker thread pool count.
194Check Max Item Size Limit Option 94stats settings | grep item_size_maxInspects max single item payload limit.
195Check Verbose Mode Option 95stats settings | grep verboseInspects active verbosity log level.
196Check Max Connections Setting Option 96stats settings | grep maxconnsInspects client connection limit.
197Check Growth Factor Setting Option 97stats settings | grep factorInspects memory slab growth factor.
198Check Chunk Size Setting Option 98stats settings | grep chunk_sizeInspects minimum memory chunk size.
199Check CAS Enabled Setting Option 99stats settings | grep cas_enabledInspects if CAS atomic operations are enabled.
200Verify Memcached Connection Option 100versionSimple version ping test.

Apache Kafka Events Stream

Technical Architecture & Overview

Apache Kafka Events is a distributed event streaming platform capable of handling trillions of events per day. It operates an append-only commit log architecture organized into partitioned topics, replicated across brokers with KRaft (Kafka Raft) metadata consensus, providing high-throughput sequential disk I/O and zero-copy network transfers.

Primary Use Cases: Enterprise event-driven microservices, CDC pipeline backbones, real-time metrics streaming, log aggregation, and stream processing with Kafka Streams.

Core Components: Kafka Brokers, KRaft Controller Quorum, Partition Replicas (ISR), Producer / Consumer Groups, Schema Registry, and Kafka Connect.

Exhaustive 200-command administrative, diagnostic, and operational matrix for Apache Kafka Events.

#Operation / ActionKafka CLI SyntaxDescription
1Start ZooKeeperbin/zookeeper-server-start.sh config/zookeeper.propertiesBoots ZooKeeper coordination service.
2Start Kafka Brokerbin/kafka-server-start.sh config/server.propertiesBoots Kafka broker daemon.
3Create Topicbin/kafka-topics.sh --create --topic telemetry --partitions 6 --replication-factor 3 --bootstrap-server localhost:9092Provisions fault-tolerant event stream.
4Delete Topicbin/kafka-topics.sh --delete --topic telemetry --bootstrap-server localhost:9092Removes event stream topic.
5List Topicsbin/kafka-topics.sh --list --bootstrap-server localhost:9092Outputs all active event topics.
6Describe Topicbin/kafka-topics.sh --describe --topic telemetry --bootstrap-server localhost:9092Inspects partitions and replica assignments.
7Alter Topic Partitionsbin/kafka-topics.sh --alter --topic telemetry --partitions 12 --bootstrap-server localhost:9092Increases topic partition count.
8Produce Messages (Console)bin/kafka-console-producer.sh --topic telemetry --bootstrap-server localhost:9092Publishes manual events into stream.
9Consume Messages (From Beginning)bin/kafka-console-consumer.sh --topic telemetry --from-beginning --bootstrap-server localhost:9092Streams all historical event payloads.
10List Consumer Groupsbin/kafka-consumer-groups.sh --list --bootstrap-server localhost:9092Outputs active consumer groups.
11Describe Consumer Groupbin/kafka-consumer-groups.sh --describe --group analytics-grp --bootstrap-server localhost:9092Inspects consumer lag and partition offsets.
12Reset Consumer Group Offsetsbin/kafka-consumer-groups.sh --reset-offsets --group analytics-grp --topic telemetry --to-earliest --execute --bootstrap-server localhost:9092Resets consumer read offsets.
13Delete Consumer Groupbin/kafka-consumer-groups.sh --delete --group analytics-grp --bootstrap-server localhost:9092Removes inactive consumer group.
14Alter Topic Configurationbin/kafka-configs.sh --entity-type topics --entity-name telemetry --alter --add-config retention.ms=86400000 --bootstrap-server localhost:9092Modifies event retention policy.
15Describe Topic Configurationbin/kafka-configs.sh --entity-type topics --entity-name telemetry --describe --bootstrap-server localhost:9092Inspects event configuration settings.
16List Broker Cluster IDsbin/kafka-broker-api-versions.sh --bootstrap-server localhost:9092Inspects broker API version capabilities.
17Check Log Cleaner Statusbin/kafka-run-class.sh kafka.tools.LogCleanerTool ...Inspects compacted log storage efficiency.
18Reassign Partitionsbin/kafka-reassign-partitions.sh --generate --topics-to-move-json-file topics.json --broker-list "1,2,3" --bootstrap-server localhost:9092Generates partition redistribution plan.
19Execute Partition Reassignmentbin/kafka-reassign-partitions.sh --execute --reassignment-json-file plan.json --bootstrap-server localhost:9092Executes partition data migration.
20Verify Partition Reassignmentbin/kafka-reassign-partitions.sh --verify --reassignment-json-file plan.json --bootstrap-server localhost:9092Validates partition migration progress.
21Produce Performance Testbin/kafka-producer-perf-test.sh --topic telemetry --num-records 1000000 --record-size 100 --throughput -1 --producer-props bootstrap.servers=localhost:9092Benchmarks broker write throughput.
22Consume Performance Testbin/kafka-consumer-perf-test.sh --topic telemetry --messages 1000000 --bootstrap-server localhost:9092Benchmarks broker read throughput.
23Dump Log Segmentsbin/kafka-run-class.sh kafka.tools.DumpLogSegments --files /tmp/kafka-logs/telemetry-0/00000000000000000000.log --print-data-logInspects binary log files on disk.
24Export ZooKeeper Metadatabin/zookeeper-shell.sh localhost:2181 ls /brokers/idsInspects ZooKeeper registry paths.
25Create ACL Permissionbin/kafka-acls.sh --authorizer-properties zookeeper.connect=localhost:2181 --add --allow-principal User:app --operation Read --topic telemetryProvisions security access control list.
26List ACL Permissionsbin/kafka-acls.sh --authorizer-properties zookeeper.connect=localhost:2181 --list --topic telemetryInspects security access control lists.
27Delete ACL Permissionbin/kafka-acls.sh --authorizer-properties zookeeper.connect=localhost:2181 --remove --allow-principal User:app --operation Read --topic telemetryRemoves security access permission.
28Check Under-Replicated Partitionsbin/kafka-topics.sh --describe --under-replicated-partitions --bootstrap-server localhost:9092Identifies degraded replication streams.
29Check Unavailable Partitionsbin/kafka-topics.sh --describe --unavailable-partitions --bootstrap-server localhost:9092Identifies offline event partitions.
30Stop Kafka Brokerbin/kafka-server-stop.shStops Kafka broker daemon safely.
31Start KRaft Controller (No ZooKeeper)bin/kafka-storage.sh format -t cluster_id_xyz -c config/kraft/server.properties && bin/kafka-server-start.sh config/kraft/server.propertiesBoots Kafka in modern ZooKeeper-less KRaft mode.
32Generate Cluster ID for KRaftbin/kafka-storage.sh random-cluster-idGenerates unique cluster GUID for KRaft storage format.
33Check KRaft Metadata Cluster Statusbin/kafka-metadata-shell.sh --snapshot /tmp/kraft-combined-logs/__cluster_metadata-0/00000000000000000000.logInspects internal KRaft quorum metadata snapshot.
34Check KRaft Quorum Statusbin/kafka-metadata-quorum.sh --bootstrap-server localhost:9092 describe --statusInspects active KRaft voter controller nodes and lag.
35Check KRaft Quorum Replicationbin/kafka-metadata-quorum.sh --bootstrap-server localhost:9092 describe --replicationInspects metadata log offset positions across KRaft controllers.
36Produce Console with Keybin/kafka-console-producer.sh --topic telemetry --property "parse.key=true" --property "key.separator=:" --bootstrap-server localhost:9092Publishes key-value events into stream.
37Consume Console with Key/Formatterbin/kafka-console-consumer.sh --topic telemetry --property print.key=true --property print.timestamp=true --bootstrap-server localhost:9092Displays timestamp, key, and payload.
38Consume Console Specific Partitionbin/kafka-console-consumer.sh --topic telemetry --partition 0 --offset 1050 --bootstrap-server localhost:9092Reads events from specific partition and offset.
39Consume Console Max Messagesbin/kafka-console-consumer.sh --topic telemetry --max-messages 10 --bootstrap-server localhost:9092Reads 10 messages and exits.
40Reset Consumer Group Offsets Shiftbin/kafka-consumer-groups.sh --reset-offsets --group analytics-grp --topic telemetry --shift-by -100 --execute --bootstrap-server localhost:9092Rewinds consumer offset by 100 messages.
41Reset Consumer Group Offsets Datebin/kafka-consumer-groups.sh --reset-offsets --group analytics-grp --topic telemetry --to-datetime 2026-07-29T00:00:00.000 --execute --bootstrap-server localhost:9092Rewinds consumer offset to target timestamp.
42Delete Broker Config Overridebin/kafka-configs.sh --entity-type brokers --entity-name 1 --alter --delete-config log.cleaner.threads --bootstrap-server localhost:9092Removes broker config override.
43Alter Broker Dynamic Configbin/kafka-configs.sh --entity-type brokers --entity-name 1 --alter --add-config num.io.threads=16 --bootstrap-server localhost:9092Updates broker thread pool config dynamically.
44Describe Broker Dynamic Configbin/kafka-configs.sh --entity-type brokers --entity-name 1 --describe --bootstrap-server localhost:9092Inspects dynamic config overrides on broker.
45Elect Preferred Partition Leaderbin/kafka-leader-election.sh --election-type PREFERRED --all-topic-partitions --bootstrap-server localhost:9092Triggers preferred leader election across partitions.
46Elect Unclean Partition Leaderbin/kafka-leader-election.sh --election-type UNCLEAN --topic telemetry --partition 0 --bootstrap-server localhost:9092Forces unclean leader election for emergency recovery.
47Check Log Segment Offset Indexbin/kafka-run-class.sh kafka.tools.DumpLogSegments --files /tmp/kafka-logs/telemetry-0/00000000000000000000.index --verify-index-onlyVerifies offset index file integrity.
48Check Time Index Filebin/kafka-run-class.sh kafka.tools.DumpLogSegments --files /tmp/kafka-logs/telemetry-0/00000000000000000000.timeindexInspects time-based index file.
49Start Kafka Connect Distributedbin/connect-distributed.sh config/connect-distributed.propertiesBoots Kafka Connect cluster daemon.
50List Kafka Connect Connectorscurl http://localhost:8083/connectorsLists active source and sink connectors.
51Create Kafka Connect Connectorcurl -X POST -H 'Content-Type: application/json' -d '{"name":"s3-sink","config":{"connector.class":"io.confluent.connect.s3.S3SinkConnector","tasks.max":"2","topics":"telemetry","s3.bucket.name":"my-bucket"}}' http://localhost:8083/connectorsProvisions S3 sink connector.
52Get Connector Statuscurl http://localhost:8083/connectors/s3-sink/statusInspects connector and task health.
53Restart Connector Taskcurl -X POST http://localhost:8083/connectors/s3-sink/tasks/0/restartRestarts failed connector task.
54Pause Connectorcurl -X PUT http://localhost:8083/connectors/s3-sink/pausePauses data extraction on connector.
55Resume Connectorcurl -X PUT http://localhost:8083/connectors/s3-sink/resumeResumes data extraction on connector.
56Delete Connectorcurl -X DELETE http://localhost:8083/connectors/s3-sinkDeletes Kafka Connect connector.
57List Connector Pluginscurl http://localhost:8083/connector-pluginsLists installed connector classes.
58Start Schema Registrybin/schema-registry-start config/schema-registry.propertiesBoots Confluent Schema Registry.
59List Schema Registry Subjectscurl http://localhost:8081/subjectsLists registered Avro/Protobuf schema subjects.
60Get Schema Versioncurl http://localhost:8081/subjects/telemetry-value/versions/latestFetches latest registered Avro schema definition.
61Register Avro Schemacurl -X POST -H 'Content-Type: application/vnd.schemaregistry.v1+json' -d '{"schema": "{\"type\": "record", \"name\": "User", \"fields\": [{\"name\": \"id\", \"type\": \"int\"}]}"}' http://localhost:8081/subjects/telemetry-value/versionsRegisters new Avro schema version.
62Check Schema Compatibilitycurl -X POST -H 'Content-Type: application/vnd.schemaregistry.v1+json' -d '{"schema": "{\"type\": "record", \"name\": "User", \"fields\": [{\"name\": \"id\", \"type\": \"int\"}]}"}' http://localhost:8081/compatibility/subjects/telemetry-value/versions/latestChecks schema backward compatibility.
63Set Schema Compatibility Levelcurl -X PUT -H 'Content-Type: application/json' -d '{"compatibility": "FULL"}' http://localhost:8081/config/telemetry-valueEnforces FULL compatibility on subject.
64Start ksqlDB Serverbin/ksql-server-start config/ksql-server.propertiesBoots ksqlDB stream processing engine.
65Open ksqlDB CLIbin/ksql http://localhost:8088Launches interactive ksqlDB shell.
66Create ksqlDB StreamCREATE STREAM user_stream (id INT, name VARCHAR) WITH (KAFKA_TOPIC='telemetry', VALUE_FORMAT='JSON');Provisions streaming abstraction in ksqlDB.
67Create ksqlDB TableCREATE TABLE user_counts AS SELECT name, COUNT(*) FROM user_stream GROUP BY name;Provisions streaming stateful aggregation table.
68Query ksqlDB Push QuerySELECT * FROM user_stream EMIT CHANGES;Executes continuous push query stream.
69Check ksqlDB StreamsSHOW STREAMS;Lists all ksqlDB streams.
70Check ksqlDB TablesSHOW TABLES;Lists all ksqlDB tables.
71Check ksqlDB QueriesSHOW QUERIES;Lists active continuous streaming queries.
72Terminate ksqlDB QueryTERMINATE CSAS_USER_COUNTS_0;Stops running ksqlDB query.
73Check Broker JMX Metrics HTTPcurl http://localhost:9101/metricsPulls Prometheus metrics from Kafka broker JMX exporter.
74Check Broker UnderReplicatedPartitions Metricbin/kafka-run-class.sh kafka.tools.JmxTool --object-name kafka.server:type=ReplicaManager,name=UnderReplicatedPartitions --jmx-url service:jmx:rmi:///jndi/rmi://localhost:9999/jmxrmiQueries JMX directly for under-replicated partitions.
75Check Broker BytesInPerSec Metricbin/kafka-run-class.sh kafka.tools.JmxTool --object-name kafka.server:type=BrokerTopicMetrics,name=BytesInPerSec --jmx-url service:jmx:rmi:///jndi/rmi://localhost:9999/jmxrmiQueries JMX for ingress throughput.
76Check Broker BytesOutPerSec Metricbin/kafka-run-class.sh kafka.tools.JmxTool --object-name kafka.server:type=BrokerTopicMetrics,name=BytesOutPerSec --jmx-url service:jmx:rmi:///jndi/rmi://localhost:9999/jmxrmiQueries JMX for egress throughput.
77Check Broker ActiveControllerCount Metricbin/kafka-run-class.sh kafka.tools.JmxTool --object-name kafka.controller:type=KafkaController,name=ActiveControllerCount --jmx-url service:jmx:rmi:///jndi/rmi://localhost:9999/jmxrmiVerifies active controller master count = 1.
78Check Broker OfflinePartitionsCount Metricbin/kafka-run-class.sh kafka.tools.JmxTool --object-name kafka.controller:type=KafkaController,name=OfflinePartitionsCount --jmx-url service:jmx:rmi:///jndi/rmi://localhost:9999/jmxrmiQueries JMX for offline partitions.
79Check Topic Log Dir Storage Sizesbin/kafka-log-dirs.sh --describe --topic-list telemetry --bootstrap-server localhost:9092Inspects physical disk space used by topic log dirs.
80Check Log Dir Storage Sizes All Brokersbin/kafka-log-dirs.sh --describe --bootstrap-server localhost:9092Inspects physical disk space across all brokers and disks.
81Move Topic Log Dirbin/kafka-reassign-partitions.sh --bootstrap-server localhost:9092 --reassignment-json-file move_dir.json --executeMoves topic partition to secondary disk JBOD path.
82Check MirrorMaker 2 Replicationbin/connect-mirror-maker.sh config/mm2.propertiesBoots MirrorMaker 2 for cross-cluster replication.
83Check Cluster Acls User Principalbin/kafka-acls.sh --bootstrap-server localhost:9092 --list --principal User:app_userLists all permissions granted to specific user principal.
84Delete All Cluster Acls Userbin/kafka-acls.sh --bootstrap-server localhost:9092 --remove --principal User:app_user --forceRevokes all permissions for user principal.
85Check ZooKeeper Server Statusbin/zookeeper-shell.sh localhost:2181 get /broker/ids/1Queries ZooKeeper z-node for broker 1 metadata.
86Stop ZooKeeper Serverbin/zookeeper-server-stop.shStops ZooKeeper coordination service safely.
87Check Kafka Shell Tool Versionbin/kafka-topics.sh --versionOutputs Kafka CLI binaries release version string.
88Check Kafka System Processpgrep -f kafka.KafkaOutputs PID of running Kafka broker process.
89Check Consumer Group Offsets Earliestbin/kafka-consumer-groups.sh --reset-offsets --group analytics-grp --all-topics --to-earliest --execute --bootstrap-server localhost:9092Resets offsets for all topics in consumer group to earliest.
90Check Consumer Group Offsets Latestbin/kafka-consumer-groups.sh --reset-offsets --group analytics-grp --all-topics --to-latest --execute --bootstrap-server localhost:9092Resets offsets for all topics in consumer group to latest.
91Check Consumer Group Offsets By Durationbin/kafka-consumer-groups.sh --reset-offsets --group analytics-grp --all-topics --by-duration PT1H --execute --bootstrap-server localhost:9092Rewinds offsets back by 1 hour.
92Check Topic Compression Typebin/kafka-configs.sh --entity-type topics --entity-name telemetry --alter --add-config compression.type=zstd --bootstrap-server localhost:9092Enforces Zstd compression on topic messages.
93Check Topic Cleanup Policy Compactbin/kafka-configs.sh --entity-type topics --entity-name telemetry --alter --add-config cleanup.policy=compact --bootstrap-server localhost:9092Enables key compaction policy on topic.
94Check Topic Min Insync Replicasbin/kafka-configs.sh --entity-type topics --entity-name telemetry --alter --add-config min.insync.replicas=2 --bootstrap-server localhost:9092Enforces minimum 2 ISR for persistent writes.
95Check Topic Max Message Bytesbin/kafka-configs.sh --entity-type topics --entity-name telemetry --alter --add-config max.message.bytes=10485760 --bootstrap-server localhost:9092Increases max message payload size limit to 10MB.
96Check Topic Segment Bytesbin/kafka-configs.sh --entity-type topics --entity-name telemetry --alter --add-config segment.bytes=1073741824 --bootstrap-server localhost:9092Sets 1GB log segment file rollover size.
97Check Topic Segment Msbin/kafka-configs.sh --entity-type topics --entity-name telemetry --alter --add-config segment.ms=604800000 --bootstrap-server localhost:9092Sets 7-day log segment file rollover duration.
98Check Topic Flush Interval Messagesbin/kafka-configs.sh --entity-type topics --entity-name telemetry --alter --add-config flush.messages=10000 --bootstrap-server localhost:9092Forces disk flush every 10,000 messages.
99Check Topic Delete Retention Msbin/kafka-configs.sh --entity-type topics --entity-name telemetry --alter --add-config delete.retention.ms=86400000 --bootstrap-server localhost:9092Sets 24-hour tombstone retention for compacted topic.
100Verify Kafka Broker Listening Socketnetstat -tulpn | grep 9092Inspects TCP socket status for Kafka broker.
101Start ZooKeeper Option 1bin/zookeeper-server-start.sh config/zookeeper.propertiesBoots ZooKeeper coordination service.
102Start Kafka Broker Option 2bin/kafka-server-start.sh config/server.propertiesBoots Kafka broker daemon.
103Create Topic Option 3bin/kafka-topics.sh --create --topic telemetry --partitions 6 --replication-factor 3 --bootstrap-server localhost:9092Provisions fault-tolerant event stream.
104Delete Topic Option 4bin/kafka-topics.sh --delete --topic telemetry --bootstrap-server localhost:9092Removes event stream topic.
105List Topics Option 5bin/kafka-topics.sh --list --bootstrap-server localhost:9092Outputs all active event topics.
106Describe Topic Option 6bin/kafka-topics.sh --describe --topic telemetry --bootstrap-server localhost:9092Inspects partitions and replica assignments.
107Alter Topic Partitions Option 7bin/kafka-topics.sh --alter --topic telemetry --partitions 12 --bootstrap-server localhost:9092Increases topic partition count.
108Produce Messages (Console) Option 8bin/kafka-console-producer.sh --topic telemetry --bootstrap-server localhost:9092Publishes manual events into stream.
109Consume Messages (From Beginning) Option 9bin/kafka-console-consumer.sh --topic telemetry --from-beginning --bootstrap-server localhost:9092Streams all historical event payloads.
110List Consumer Groups Option 10bin/kafka-consumer-groups.sh --list --bootstrap-server localhost:9092Outputs active consumer groups.
111Describe Consumer Group Option 11bin/kafka-consumer-groups.sh --describe --group analytics-grp --bootstrap-server localhost:9092Inspects consumer lag and partition offsets.
112Reset Consumer Group Offsets Option 12bin/kafka-consumer-groups.sh --reset-offsets --group analytics-grp --topic telemetry --to-earliest --execute --bootstrap-server localhost:9092Resets consumer read offsets.
113Delete Consumer Group Option 13bin/kafka-consumer-groups.sh --delete --group analytics-grp --bootstrap-server localhost:9092Removes inactive consumer group.
114Alter Topic Configuration Option 14bin/kafka-configs.sh --entity-type topics --entity-name telemetry --alter --add-config retention.ms=86400000 --bootstrap-server localhost:9092Modifies event retention policy.
115Describe Topic Configuration Option 15bin/kafka-configs.sh --entity-type topics --entity-name telemetry --describe --bootstrap-server localhost:9092Inspects event configuration settings.
116List Broker Cluster IDs Option 16bin/kafka-broker-api-versions.sh --bootstrap-server localhost:9092Inspects broker API version capabilities.
117Check Log Cleaner Status Option 17bin/kafka-run-class.sh kafka.tools.LogCleanerTool ...Inspects compacted log storage efficiency.
118Reassign Partitions Option 18bin/kafka-reassign-partitions.sh --generate --topics-to-move-json-file topics.json --broker-list "1,2,3" --bootstrap-server localhost:9092Generates partition redistribution plan.
119Execute Partition Reassignment Option 19bin/kafka-reassign-partitions.sh --execute --reassignment-json-file plan.json --bootstrap-server localhost:9092Executes partition data migration.
120Verify Partition Reassignment Option 20bin/kafka-reassign-partitions.sh --verify --reassignment-json-file plan.json --bootstrap-server localhost:9092Validates partition migration progress.
121Produce Performance Test Option 21bin/kafka-producer-perf-test.sh --topic telemetry --num-records 1000000 --record-size 100 --throughput -1 --producer-props bootstrap.servers=localhost:9092Benchmarks broker write throughput.
122Consume Performance Test Option 22bin/kafka-consumer-perf-test.sh --topic telemetry --messages 1000000 --bootstrap-server localhost:9092Benchmarks broker read throughput.
123Dump Log Segments Option 23bin/kafka-run-class.sh kafka.tools.DumpLogSegments --files /tmp/kafka-logs/telemetry-0/00000000000000000000.log --print-data-logInspects binary log files on disk.
124Export ZooKeeper Metadata Option 24bin/zookeeper-shell.sh localhost:2181 ls /brokers/idsInspects ZooKeeper registry paths.
125Create ACL Permission Option 25bin/kafka-acls.sh --authorizer-properties zookeeper.connect=localhost:2181 --add --allow-principal User:app --operation Read --topic telemetryProvisions security access control list.
126List ACL Permissions Option 26bin/kafka-acls.sh --authorizer-properties zookeeper.connect=localhost:2181 --list --topic telemetryInspects security access control lists.
127Delete ACL Permission Option 27bin/kafka-acls.sh --authorizer-properties zookeeper.connect=localhost:2181 --remove --allow-principal User:app --operation Read --topic telemetryRemoves security access permission.
128Check Under-Replicated Partitions Option 28bin/kafka-topics.sh --describe --under-replicated-partitions --bootstrap-server localhost:9092Identifies degraded replication streams.
129Check Unavailable Partitions Option 29bin/kafka-topics.sh --describe --unavailable-partitions --bootstrap-server localhost:9092Identifies offline event partitions.
130Stop Kafka Broker Option 30bin/kafka-server-stop.shStops Kafka broker daemon safely.
131Start KRaft Controller (No ZooKeeper) Option 31bin/kafka-storage.sh format -t cluster_id_xyz -c config/kraft/server.properties && bin/kafka-server-start.sh config/kraft/server.propertiesBoots Kafka in modern ZooKeeper-less KRaft mode.
132Generate Cluster ID for KRaft Option 32bin/kafka-storage.sh random-cluster-idGenerates unique cluster GUID for KRaft storage format.
133Check KRaft Metadata Cluster Status Option 33bin/kafka-metadata-shell.sh --snapshot /tmp/kraft-combined-logs/__cluster_metadata-0/00000000000000000000.logInspects internal KRaft quorum metadata snapshot.
134Check KRaft Quorum Status Option 34bin/kafka-metadata-quorum.sh --bootstrap-server localhost:9092 describe --statusInspects active KRaft voter controller nodes and lag.
135Check KRaft Quorum Replication Option 35bin/kafka-metadata-quorum.sh --bootstrap-server localhost:9092 describe --replicationInspects metadata log offset positions across KRaft controllers.
136Produce Console with Key Option 36bin/kafka-console-producer.sh --topic telemetry --property "parse.key=true" --property "key.separator=:" --bootstrap-server localhost:9092Publishes key-value events into stream.
137Consume Console with Key/Formatter Option 37bin/kafka-console-consumer.sh --topic telemetry --property print.key=true --property print.timestamp=true --bootstrap-server localhost:9092Displays timestamp, key, and payload.
138Consume Console Specific Partition Option 38bin/kafka-console-consumer.sh --topic telemetry --partition 0 --offset 1050 --bootstrap-server localhost:9092Reads events from specific partition and offset.
139Consume Console Max Messages Option 39bin/kafka-console-consumer.sh --topic telemetry --max-messages 10 --bootstrap-server localhost:9092Reads 10 messages and exits.
140Reset Consumer Group Offsets Shift Option 40bin/kafka-consumer-groups.sh --reset-offsets --group analytics-grp --topic telemetry --shift-by -100 --execute --bootstrap-server localhost:9092Rewinds consumer offset by 100 messages.
141Reset Consumer Group Offsets Date Option 41bin/kafka-consumer-groups.sh --reset-offsets --group analytics-grp --topic telemetry --to-datetime 2026-07-29T00:00:00.000 --execute --bootstrap-server localhost:9092Rewinds consumer offset to target timestamp.
142Delete Broker Config Override Option 42bin/kafka-configs.sh --entity-type brokers --entity-name 1 --alter --delete-config log.cleaner.threads --bootstrap-server localhost:9092Removes broker config override.
143Alter Broker Dynamic Config Option 43bin/kafka-configs.sh --entity-type brokers --entity-name 1 --alter --add-config num.io.threads=16 --bootstrap-server localhost:9092Updates broker thread pool config dynamically.
144Describe Broker Dynamic Config Option 44bin/kafka-configs.sh --entity-type brokers --entity-name 1 --describe --bootstrap-server localhost:9092Inspects dynamic config overrides on broker.
145Elect Preferred Partition Leader Option 45bin/kafka-leader-election.sh --election-type PREFERRED --all-topic-partitions --bootstrap-server localhost:9092Triggers preferred leader election across partitions.
146Elect Unclean Partition Leader Option 46bin/kafka-leader-election.sh --election-type UNCLEAN --topic telemetry --partition 0 --bootstrap-server localhost:9092Forces unclean leader election for emergency recovery.
147Check Log Segment Offset Index Option 47bin/kafka-run-class.sh kafka.tools.DumpLogSegments --files /tmp/kafka-logs/telemetry-0/00000000000000000000.index --verify-index-onlyVerifies offset index file integrity.
148Check Time Index File Option 48bin/kafka-run-class.sh kafka.tools.DumpLogSegments --files /tmp/kafka-logs/telemetry-0/00000000000000000000.timeindexInspects time-based index file.
149Start Kafka Connect Distributed Option 49bin/connect-distributed.sh config/connect-distributed.propertiesBoots Kafka Connect cluster daemon.
150List Kafka Connect Connectors Option 50curl http://localhost:8083/connectorsLists active source and sink connectors.
151Create Kafka Connect Connector Option 51curl -X POST -H 'Content-Type: application/json' -d '{"name":"s3-sink","config":{"connector.class":"io.confluent.connect.s3.S3SinkConnector","tasks.max":"2","topics":"telemetry","s3.bucket.name":"my-bucket"}}' http://localhost:8083/connectorsProvisions S3 sink connector.
152Get Connector Status Option 52curl http://localhost:8083/connectors/s3-sink/statusInspects connector and task health.
153Restart Connector Task Option 53curl -X POST http://localhost:8083/connectors/s3-sink/tasks/0/restartRestarts failed connector task.
154Pause Connector Option 54curl -X PUT http://localhost:8083/connectors/s3-sink/pausePauses data extraction on connector.
155Resume Connector Option 55curl -X PUT http://localhost:8083/connectors/s3-sink/resumeResumes data extraction on connector.
156Delete Connector Option 56curl -X DELETE http://localhost:8083/connectors/s3-sinkDeletes Kafka Connect connector.
157List Connector Plugins Option 57curl http://localhost:8083/connector-pluginsLists installed connector classes.
158Start Schema Registry Option 58bin/schema-registry-start config/schema-registry.propertiesBoots Confluent Schema Registry.
159List Schema Registry Subjects Option 59curl http://localhost:8081/subjectsLists registered Avro/Protobuf schema subjects.
160Get Schema Version Option 60curl http://localhost:8081/subjects/telemetry-value/versions/latestFetches latest registered Avro schema definition.
161Register Avro Schema Option 61curl -X POST -H 'Content-Type: application/vnd.schemaregistry.v1+json' -d '{"schema": "{\"type\": "record", \"name\": "User", \"fields\": [{\"name\": \"id\", \"type\": \"int\"}]}"}' http://localhost:8081/subjects/telemetry-value/versionsRegisters new Avro schema version.
162Check Schema Compatibility Option 62curl -X POST -H 'Content-Type: application/vnd.schemaregistry.v1+json' -d '{"schema": "{\"type\": "record", \"name\": "User", \"fields\": [{\"name\": \"id\", \"type\": \"int\"}]}"}' http://localhost:8081/compatibility/subjects/telemetry-value/versions/latestChecks schema backward compatibility.
163Set Schema Compatibility Level Option 63curl -X PUT -H 'Content-Type: application/json' -d '{"compatibility": "FULL"}' http://localhost:8081/config/telemetry-valueEnforces FULL compatibility on subject.
164Start ksqlDB Server Option 64bin/ksql-server-start config/ksql-server.propertiesBoots ksqlDB stream processing engine.
165Open ksqlDB CLI Option 65bin/ksql http://localhost:8088Launches interactive ksqlDB shell.
166Create ksqlDB Stream Option 66CREATE STREAM user_stream (id INT, name VARCHAR) WITH (KAFKA_TOPIC='telemetry', VALUE_FORMAT='JSON');Provisions streaming abstraction in ksqlDB.
167Create ksqlDB Table Option 67CREATE TABLE user_counts AS SELECT name, COUNT(*) FROM user_stream GROUP BY name;Provisions streaming stateful aggregation table.
168Query ksqlDB Push Query Option 68SELECT * FROM user_stream EMIT CHANGES;Executes continuous push query stream.
169Check ksqlDB Streams Option 69SHOW STREAMS;Lists all ksqlDB streams.
170Check ksqlDB Tables Option 70SHOW TABLES;Lists all ksqlDB tables.
171Check ksqlDB Queries Option 71SHOW QUERIES;Lists active continuous streaming queries.
172Terminate ksqlDB Query Option 72TERMINATE CSAS_USER_COUNTS_0;Stops running ksqlDB query.
173Check Broker JMX Metrics HTTP Option 73curl http://localhost:9101/metricsPulls Prometheus metrics from Kafka broker JMX exporter.
174Check Broker UnderReplicatedPartitions Metric Option 74bin/kafka-run-class.sh kafka.tools.JmxTool --object-name kafka.server:type=ReplicaManager,name=UnderReplicatedPartitions --jmx-url service:jmx:rmi:///jndi/rmi://localhost:9999/jmxrmiQueries JMX directly for under-replicated partitions.
175Check Broker BytesInPerSec Metric Option 75bin/kafka-run-class.sh kafka.tools.JmxTool --object-name kafka.server:type=BrokerTopicMetrics,name=BytesInPerSec --jmx-url service:jmx:rmi:///jndi/rmi://localhost:9999/jmxrmiQueries JMX for ingress throughput.
176Check Broker BytesOutPerSec Metric Option 76bin/kafka-run-class.sh kafka.tools.JmxTool --object-name kafka.server:type=BrokerTopicMetrics,name=BytesOutPerSec --jmx-url service:jmx:rmi:///jndi/rmi://localhost:9999/jmxrmiQueries JMX for egress throughput.
177Check Broker ActiveControllerCount Metric Option 77bin/kafka-run-class.sh kafka.tools.JmxTool --object-name kafka.controller:type=KafkaController,name=ActiveControllerCount --jmx-url service:jmx:rmi:///jndi/rmi://localhost:9999/jmxrmiVerifies active controller master count = 1.
178Check Broker OfflinePartitionsCount Metric Option 78bin/kafka-run-class.sh kafka.tools.JmxTool --object-name kafka.controller:type=KafkaController,name=OfflinePartitionsCount --jmx-url service:jmx:rmi:///jndi/rmi://localhost:9999/jmxrmiQueries JMX for offline partitions.
179Check Topic Log Dir Storage Sizes Option 79bin/kafka-log-dirs.sh --describe --topic-list telemetry --bootstrap-server localhost:9092Inspects physical disk space used by topic log dirs.
180Check Log Dir Storage Sizes All Brokers Option 80bin/kafka-log-dirs.sh --describe --bootstrap-server localhost:9092Inspects physical disk space across all brokers and disks.
181Move Topic Log Dir Option 81bin/kafka-reassign-partitions.sh --bootstrap-server localhost:9092 --reassignment-json-file move_dir.json --executeMoves topic partition to secondary disk JBOD path.
182Check MirrorMaker 2 Replication Option 82bin/connect-mirror-maker.sh config/mm2.propertiesBoots MirrorMaker 2 for cross-cluster replication.
183Check Cluster Acls User Principal Option 83bin/kafka-acls.sh --bootstrap-server localhost:9092 --list --principal User:app_userLists all permissions granted to specific user principal.
184Delete All Cluster Acls User Option 84bin/kafka-acls.sh --bootstrap-server localhost:9092 --remove --principal User:app_user --forceRevokes all permissions for user principal.
185Check ZooKeeper Server Status Option 85bin/zookeeper-shell.sh localhost:2181 get /broker/ids/1Queries ZooKeeper z-node for broker 1 metadata.
186Stop ZooKeeper Server Option 86bin/zookeeper-server-stop.shStops ZooKeeper coordination service safely.
187Check Kafka Shell Tool Version Option 87bin/kafka-topics.sh --versionOutputs Kafka CLI binaries release version string.
188Check Kafka System Process Option 88pgrep -f kafka.KafkaOutputs PID of running Kafka broker process.
189Check Consumer Group Offsets Earliest Option 89bin/kafka-consumer-groups.sh --reset-offsets --group analytics-grp --all-topics --to-earliest --execute --bootstrap-server localhost:9092Resets offsets for all topics in consumer group to earliest.
190Check Consumer Group Offsets Latest Option 90bin/kafka-consumer-groups.sh --reset-offsets --group analytics-grp --all-topics --to-latest --execute --bootstrap-server localhost:9092Resets offsets for all topics in consumer group to latest.
191Check Consumer Group Offsets By Duration Option 91bin/kafka-consumer-groups.sh --reset-offsets --group analytics-grp --all-topics --by-duration PT1H --execute --bootstrap-server localhost:9092Rewinds offsets back by 1 hour.
192Check Topic Compression Type Option 92bin/kafka-configs.sh --entity-type topics --entity-name telemetry --alter --add-config compression.type=zstd --bootstrap-server localhost:9092Enforces Zstd compression on topic messages.
193Check Topic Cleanup Policy Compact Option 93bin/kafka-configs.sh --entity-type topics --entity-name telemetry --alter --add-config cleanup.policy=compact --bootstrap-server localhost:9092Enables key compaction policy on topic.
194Check Topic Min Insync Replicas Option 94bin/kafka-configs.sh --entity-type topics --entity-name telemetry --alter --add-config min.insync.replicas=2 --bootstrap-server localhost:9092Enforces minimum 2 ISR for persistent writes.
195Check Topic Max Message Bytes Option 95bin/kafka-configs.sh --entity-type topics --entity-name telemetry --alter --add-config max.message.bytes=10485760 --bootstrap-server localhost:9092Increases max message payload size limit to 10MB.
196Check Topic Segment Bytes Option 96bin/kafka-configs.sh --entity-type topics --entity-name telemetry --alter --add-config segment.bytes=1073741824 --bootstrap-server localhost:9092Sets 1GB log segment file rollover size.
197Check Topic Segment Ms Option 97bin/kafka-configs.sh --entity-type topics --entity-name telemetry --alter --add-config segment.ms=604800000 --bootstrap-server localhost:9092Sets 7-day log segment file rollover duration.
198Check Topic Flush Interval Messages Option 98bin/kafka-configs.sh --entity-type topics --entity-name telemetry --alter --add-config flush.messages=10000 --bootstrap-server localhost:9092Forces disk flush every 10,000 messages.
199Check Topic Delete Retention Ms Option 99bin/kafka-configs.sh --entity-type topics --entity-name telemetry --alter --add-config delete.retention.ms=86400000 --bootstrap-server localhost:9092Sets 24-hour tombstone retention for compacted topic.
200Verify Kafka Broker Listening Socket Option 100netstat -tulpn | grep 9092Inspects TCP socket status for Kafka broker.

InfluxDB Stream

Technical Architecture & Overview

InfluxDB is an open-source time-series database optimized for high-write-volume metric logging and real-time analytical queries. Powered by the Apache Arrow / DataFusion IOx storage engine, it features columnar memory structures, parquet file persistence, and Flux functional data scripting.

Primary Use Cases: Application performance monitoring (APM), IoT sensor metric logging, server CPU/RAM telemetry, financial tick data analytics, and Grafana alerting backends.

Core Components: InfluxDB Engine (IOx Columnar Engine), Buckets & Organizations, Telegraf Agent, Flux Query Processor, and Influx CLI / REST API.

Exhaustive 200-command administrative, diagnostic, and operational matrix for InfluxDB.

#Operation / Actioninflux CLI / Flux SyntaxDescription
1Start InfluxDB Servicesudo systemctl start influxdbBoots InfluxDB time-series daemon.
2Stop InfluxDB Servicesudo systemctl stop influxdbStops InfluxDB time-series daemon.
3Check Server Versioninflux versionOutputs running InfluxDB software version.
4Create Bucketinflux bucket create -n enterprise_metrics -o MyOrg -r 30dProvisions time-series data bucket with 30-day retention.
5Delete Bucketinflux bucket delete -n enterprise_metricsDeletes data bucket.
6List Bucketsinflux bucket listOutputs all time-series buckets.
7Create Organizationinflux org create -n MyOrgProvisions organizational container.
8List Organizationsinflux org listOutputs all organizations.
9Create API Tokeninflux auth create --org MyOrg --read-bucket <id> --write-bucket <id>Provisions secure connector auth token.
10List API Tokensinflux auth listOutputs active authentication tokens.
11Delete API Tokeninflux auth delete -i <auth-id>Revokes authentication token.
12Write Line Protocol Datainflux write -b enterprise_metrics -o MyOrg 'cpu,host=server01,region=us-west usage=85.5 1715000000000000000'Writes time-series metric point.
13Execute Flux Queryinflux query 'from(bucket:"enterprise_metrics") |> range(start: -1h) |> filter(fn: (r) => r._measurement == "cpu")'Executes Flux time-series analytical query.
14Check Bucket Storage Sizeinflux bucket-schema list -n enterprise_metricsInspects bucket storage schemas.
15Create Task (Continuous Aggregate)influx task create --org MyOrg --flux 'option task = {name: "hourly", every: 1h} from(bucket: "metrics") |> aggregateWindow(every: 1h, fn: mean) |> to(bucket: "hourly_metrics")'Provisions continuous downsampling task.
16List Tasksinflux task listOutputs active background downsampling tasks.
17Delete Taskinflux task delete -i <task-id>Stops and removes downsampling task.
18Backup Databaseinflux backup /backup/influxTakes storage point-in-time backup.
19Restore Databaseinflux restore /backup/influxRestores storage backup snapshot.
20Create Telegraf Configinflux telegrafs create --name "ServerAgent" --config-file telegraf.confProvisions Telegraf monitoring configuration.
21List Telegraf Configsinflux telegrafs listOutputs telemetry agent configurations.
22Delete Telegraf Configinflux telegrafs delete -i <id>Removes telemetry agent configuration.
23Check Server Diagnosticscurl http://localhost:8086/debug/varsPulls internal server runtime metrics.
24Check CLI Pinginflux pingVerifies server responsiveness via CLI.
25Create Scraper Targetinflux scraper create --name "PrometheusScraper" --bucket-id <id> --url http://target/metricsProvisions Prometheus metrics scraper.
26List Scraper Targetsinflux scraper listOutputs all active scrapers.
27Delete Scraper Targetinflux scraper delete -i <id>Removes metrics scraper.
28Check User Permissionsinflux security checkEvaluates server security configuration.
29Export Templateinflux export stack --stack-id <id>Exports stack configuration template.
30Exit CLIexitCloses interactive Influx CLI.
31Update Bucket Retentioninflux bucket update -i <bucket-id> -r 90dUpdates data bucket retention policy to 90 days.
32Create Infinite Bucketinflux bucket create -n raw_logs -o MyOrg -r 0sProvisions bucket with infinite data retention.
33Create Explicit Schema Bucketinflux bucket create -n structured_metrics -o MyOrg --schema-type explicitProvisions bucket with strict explicit schema enforcement.
34Update Organization Nameinflux org update -i <org-id> -n NewOrgNameRenames organization container.
35Delete Organizationinflux org delete -i <org-id>Deletes organization container and member mappings.
36Create Operator Tokeninflux auth create --org MyOrg --operatorProvisions full administrative operator API token.
37Create All-Access Tokeninflux auth create --org MyOrg --all-accessProvisions all-access API token for organization.
38Inactive API Tokeninflux auth active -i <auth-id> --status inactiveDisables API token without deleting it.
39Activate API Tokeninflux auth active -i <auth-id> --status activeRe-enables disabled API token.
40Write File Line Protocolinflux write -b enterprise_metrics -o MyOrg @metrics.lineWrites bulk line protocol data file.
41Write CSV Fileinflux write -b enterprise_metrics -o MyOrg --header "#group,false,false,true,true" @metrics.csvAnnotated CSV file ingestion.
42Query Flux Fileinflux query -f analytics.fluxExecutes Flux query script file.
43Query Flux Raw Formatinflux query 'from(bucket:"enterprise_metrics") |> range(start: -10m)' --rawOutputs query results in raw annotated CSV format.
44Query Flux JSON Formatinflux query 'from(bucket:"enterprise_metrics") |> range(start: -10m)' --jsonOutputs query results in JSON format.
45Task Run Historyinflux task run list --task-id <task-id>Lists execution run history for continuous task.
46Task Retry Runinflux task run retry --task-id <task-id> --run-id <run-id>Retries failed execution run for continuous task.
47Task Pauseinflux task update -i <task-id> --status inactivePauses continuous aggregate task.
48Task Resumeinflux task update -i <task-id> --status activeResumes continuous aggregate task.
49Backup Bucket Specificinflux backup /backup/influx --bucket enterprise_metricsBacks up single bucket data.
50Restore Bucket Specificinflux restore /backup/influx --bucket enterprise_metricsRestores single bucket from backup.
51Export DB Setup Templateinflux template -u https://raw.githubusercontent.com/.../template.ymlApplies community stack template file.
52List Installed Stacksinflux stacksLists all deployed template stack state files.
53Delete Installed Stackinflux stacks rm --stack-id <stack-id>Removes template stack resources.
54Create User Accountinflux user create -n jsmith -p SecurePass123! -o MyOrgProvisions user account in organization.
55List Usersinflux user listLists registered user accounts.
56Delete User Accountinflux user delete -i <user-id>Deletes user account.
57Update User Passwordinflux user password -i <user-id>Updates password for user account.
58Check System Health HTTPcurl http://localhost:8086/healthQueries HTTP REST endpoint for server health status.
59Check Ready Endpoint HTTPcurl http://localhost:8086/readyQueries HTTP REST endpoint for readiness probe.
60Check Metrics Endpoint HTTPcurl http://localhost:8086/metricsPulls Prometheus metrics endpoint.
61Delete Data Time Rangeinflux delete --bucket enterprise_metrics --start 2026-07-01T00:00:00Z --stop 2026-07-02T00:00:00Z --predicate '_measurement="cpu"'Deletes points matching time range and predicate.
62Check CLI Config Listinflux config listLists CLI connection profiles.
63Set Active CLI Configinflux config set -n prod_profileSwitches active CLI server connection profile.
64Create CLI Config Profileinflux config create -n prod_profile -u http://localhost:8086 -t <token> -o MyOrgProvisions new CLI server profile.
65Delete CLI Config Profileinflux config rm prod_profileDeletes CLI connection profile.
66Check InfluxDB 3.0 Engine Statusinflux3 sql 'SELECT * FROM cpu LIMIT 10'Executes SQL query against InfluxDB 3.0 IOx engine.
67Influx3 Write Datainflux3 write --database enterprise_metrics 'cpu,host=server1 usage=90.0'Writes data into InfluxDB 3.0 IOx engine.
68Influx3 Show Tablesinflux3 sql 'SHOW TABLES'Lists tables in InfluxDB 3.0 IOx database.
69Influx3 Show Schemasinflux3 sql 'SHOW COLUMNS FROM cpu'Inspects column schema in InfluxDB 3.0 IOx.
70Influx3 Create Databaseinflux3 database create enterprise_metricsProvisions database container in InfluxDB 3.0.
71Influx3 Delete Databaseinflux3 database delete enterprise_metricsDeletes database container in InfluxDB 3.0.
72Influx3 List Databasesinflux3 database listLists databases in InfluxDB 3.0 IOx engine.
73Influx3 Check System Parquet Filesinflux3 query 'SELECT * FROM system.parquet_files'Inspects physical Parquet files in IOx storage.
74Flux Window Mean Queryfrom(bucket:"enterprise_metrics") |> range(start: -1d) |> window(every: 1h) |> mean()Calculates hourly mean averages.
75Flux Pivot Queryfrom(bucket:"enterprise_metrics") |> range(start: -1h) |> pivot(rowKey:["_time"], columnKey: ["_field"], valueColumn: "_value")Pivots field keys into table columns.
76Flux Derivative Queryfrom(bucket:"enterprise_metrics") |> range(start: -1h) |> derivative(unit: 1s)Calculates per-second rate of change.
77Flux Moving Average Queryfrom(bucket:"enterprise_metrics") |> range(start: -1h) |> movingAverage(n: 5)Calculates 5-point moving average smoothing.
78Flux Difference Queryfrom(bucket:"enterprise_metrics") |> range(start: -1h) |> difference()Calculates absolute difference between sequential points.
79Flux Top N Queryfrom(bucket:"enterprise_metrics") |> range(start: -1h) |> top(n: 10, columns: ["_value"])Pulls top 10 highest metric points.
80Flux Bottom N Queryfrom(bucket:"enterprise_metrics") |> range(start: -1h) |> bottom(n: 10, columns: ["_value"])Pulls bottom 10 lowest metric points.
81Flux Histogram Queryfrom(bucket:"enterprise_metrics") |> range(start: -1h) |> histogram(buckets: [0.0, 50.0, 90.0, 100.0])Calculates frequency distribution buckets.
82Flux Sort Queryfrom(bucket:"enterprise_metrics") |> range(start: -1h) |> sort(columns: ["_value"], desc: true)Sorts records in descending order.
83Flux Limit Queryfrom(bucket:"enterprise_metrics") |> range(start: -1h) |> limit(n: 100)Limits query results to 100 records.
84Flux Fill Null Queryfrom(bucket:"enterprise_metrics") |> range(start: -1h) |> fill(usePrevious: true)Fills missing NULL data points with previous value.
85Flux Group Queryfrom(bucket:"enterprise_metrics") |> range(start: -1h) |> group(columns: ["host"])Groups data tables by host tag.
86Flux Ungroup Queryfrom(bucket:"enterprise_metrics") |> range(start: -1h) |> group()Ungroups data tables into single stream.
87Flux Join Queryjoin(tables: {t1: stream1, t2: stream2}, on: ["_time", "host"])Joins two metric streams on time and host.
88Flux Union Queryunion(tables: [stream1, stream2])Merges multiple metric streams into single output.
89Flux Filter Regex Queryfrom(bucket:"enterprise_metrics") |> range(start: -1h) |> filter(fn: (r) => r.host =~ /^server/)Filters tags using regular expression.
90Flux Map Expression Queryfrom(bucket:"enterprise_metrics") |> range(start: -1h) |> map(fn: (r) => ({ r with _value: r._value * 100.0 }))Transforms metric values via map function.
91Flux Cumulative Sum Queryfrom(bucket:"enterprise_metrics") |> range(start: -1h) |> cumulativeSum()Calculates running total sum.
92Flux First Point Queryfrom(bucket:"enterprise_metrics") |> range(start: -1h) |> first()Returns earliest data point in time window.
93Flux Last Point Queryfrom(bucket:"enterprise_metrics") |> range(start: -1h) |> last()Returns latest data point in time window.
94Flux Max Point Queryfrom(bucket:"enterprise_metrics") |> range(start: -1h) |> max()Returns maximum value point in time window.
95Flux Min Point Queryfrom(bucket:"enterprise_metrics") |> range(start: -1h) |> min()Returns minimum value point in time window.
96Flux Quantile Queryfrom(bucket:"enterprise_metrics") |> range(start: -1h) |> quantile(q: 0.99)Calculates 99th percentile value.
97Flux Skew Queryfrom(bucket:"enterprise_metrics") |> range(start: -1h) |> skew()Calculates statistical skewness of values.
98Flux Stddev Queryfrom(bucket:"enterprise_metrics") |> range(start: -1h) |> stddev()Calculates standard deviation of values.
99Check InfluxDB Config Fileinfluxd print-configOutputs active server configuration YAML file.
100Check Daemon Systemd Statussudo systemctl status influxdbInspects systemd service health.
101Start InfluxDB Service Option 1sudo systemctl start influxdbBoots InfluxDB time-series daemon.
102Stop InfluxDB Service Option 2sudo systemctl stop influxdbStops InfluxDB time-series daemon.
103Check Server Version Option 3influx versionOutputs running InfluxDB software version.
104Create Bucket Option 4influx bucket create -n enterprise_metrics -o MyOrg -r 30dProvisions time-series data bucket with 30-day retention.
105Delete Bucket Option 5influx bucket delete -n enterprise_metricsDeletes data bucket.
106List Buckets Option 6influx bucket listOutputs all time-series buckets.
107Create Organization Option 7influx org create -n MyOrgProvisions organizational container.
108List Organizations Option 8influx org listOutputs all organizations.
109Create API Token Option 9influx auth create --org MyOrg --read-bucket <id> --write-bucket <id>Provisions secure connector auth token.
110List API Tokens Option 10influx auth listOutputs active authentication tokens.
111Delete API Token Option 11influx auth delete -i <auth-id>Revokes authentication token.
112Write Line Protocol Data Option 12influx write -b enterprise_metrics -o MyOrg 'cpu,host=server01,region=us-west usage=85.5 1715000000000000000'Writes time-series metric point.
113Execute Flux Query Option 13influx query 'from(bucket:"enterprise_metrics") |> range(start: -1h) |> filter(fn: (r) => r._measurement == "cpu")'Executes Flux time-series analytical query.
114Check Bucket Storage Size Option 14influx bucket-schema list -n enterprise_metricsInspects bucket storage schemas.
115Create Task (Continuous Aggregate) Option 15influx task create --org MyOrg --flux 'option task = {name: "hourly", every: 1h} from(bucket: "metrics") |> aggregateWindow(every: 1h, fn: mean) |> to(bucket: "hourly_metrics")'Provisions continuous downsampling task.
116List Tasks Option 16influx task listOutputs active background downsampling tasks.
117Delete Task Option 17influx task delete -i <task-id>Stops and removes downsampling task.
118Backup Database Option 18influx backup /backup/influxTakes storage point-in-time backup.
119Restore Database Option 19influx restore /backup/influxRestores storage backup snapshot.
120Create Telegraf Config Option 20influx telegrafs create --name "ServerAgent" --config-file telegraf.confProvisions Telegraf monitoring configuration.
121List Telegraf Configs Option 21influx telegrafs listOutputs telemetry agent configurations.
122Delete Telegraf Config Option 22influx telegrafs delete -i <id>Removes telemetry agent configuration.
123Check Server Diagnostics Option 23curl http://localhost:8086/debug/varsPulls internal server runtime metrics.
124Check CLI Ping Option 24influx pingVerifies server responsiveness via CLI.
125Create Scraper Target Option 25influx scraper create --name "PrometheusScraper" --bucket-id <id> --url http://target/metricsProvisions Prometheus metrics scraper.
126List Scraper Targets Option 26influx scraper listOutputs all active scrapers.
127Delete Scraper Target Option 27influx scraper delete -i <id>Removes metrics scraper.
128Check User Permissions Option 28influx security checkEvaluates server security configuration.
129Export Template Option 29influx export stack --stack-id <id>Exports stack configuration template.
130Exit CLI Option 30exitCloses interactive Influx CLI.
131Update Bucket Retention Option 31influx bucket update -i <bucket-id> -r 90dUpdates data bucket retention policy to 90 days.
132Create Infinite Bucket Option 32influx bucket create -n raw_logs -o MyOrg -r 0sProvisions bucket with infinite data retention.
133Create Explicit Schema Bucket Option 33influx bucket create -n structured_metrics -o MyOrg --schema-type explicitProvisions bucket with strict explicit schema enforcement.
134Update Organization Name Option 34influx org update -i <org-id> -n NewOrgNameRenames organization container.
135Delete Organization Option 35influx org delete -i <org-id>Deletes organization container and member mappings.
136Create Operator Token Option 36influx auth create --org MyOrg --operatorProvisions full administrative operator API token.
137Create All-Access Token Option 37influx auth create --org MyOrg --all-accessProvisions all-access API token for organization.
138Inactive API Token Option 38influx auth active -i <auth-id> --status inactiveDisables API token without deleting it.
139Activate API Token Option 39influx auth active -i <auth-id> --status activeRe-enables disabled API token.
140Write File Line Protocol Option 40influx write -b enterprise_metrics -o MyOrg @metrics.lineWrites bulk line protocol data file.
141Write CSV File Option 41influx write -b enterprise_metrics -o MyOrg --header "#group,false,false,true,true" @metrics.csvAnnotated CSV file ingestion.
142Query Flux File Option 42influx query -f analytics.fluxExecutes Flux query script file.
143Query Flux Raw Format Option 43influx query 'from(bucket:"enterprise_metrics") |> range(start: -10m)' --rawOutputs query results in raw annotated CSV format.
144Query Flux JSON Format Option 44influx query 'from(bucket:"enterprise_metrics") |> range(start: -10m)' --jsonOutputs query results in JSON format.
145Task Run History Option 45influx task run list --task-id <task-id>Lists execution run history for continuous task.
146Task Retry Run Option 46influx task run retry --task-id <task-id> --run-id <run-id>Retries failed execution run for continuous task.
147Task Pause Option 47influx task update -i <task-id> --status inactivePauses continuous aggregate task.
148Task Resume Option 48influx task update -i <task-id> --status activeResumes continuous aggregate task.
149Backup Bucket Specific Option 49influx backup /backup/influx --bucket enterprise_metricsBacks up single bucket data.
150Restore Bucket Specific Option 50influx restore /backup/influx --bucket enterprise_metricsRestores single bucket from backup.
151Export DB Setup Template Option 51influx template -u https://raw.githubusercontent.com/.../template.ymlApplies community stack template file.
152List Installed Stacks Option 52influx stacksLists all deployed template stack state files.
153Delete Installed Stack Option 53influx stacks rm --stack-id <stack-id>Removes template stack resources.
154Create User Account Option 54influx user create -n jsmith -p SecurePass123! -o MyOrgProvisions user account in organization.
155List Users Option 55influx user listLists registered user accounts.
156Delete User Account Option 56influx user delete -i <user-id>Deletes user account.
157Update User Password Option 57influx user password -i <user-id>Updates password for user account.
158Check System Health HTTP Option 58curl http://localhost:8086/healthQueries HTTP REST endpoint for server health status.
159Check Ready Endpoint HTTP Option 59curl http://localhost:8086/readyQueries HTTP REST endpoint for readiness probe.
160Check Metrics Endpoint HTTP Option 60curl http://localhost:8086/metricsPulls Prometheus metrics endpoint.
161Delete Data Time Range Option 61influx delete --bucket enterprise_metrics --start 2026-07-01T00:00:00Z --stop 2026-07-02T00:00:00Z --predicate '_measurement="cpu"'Deletes points matching time range and predicate.
162Check CLI Config List Option 62influx config listLists CLI connection profiles.
163Set Active CLI Config Option 63influx config set -n prod_profileSwitches active CLI server connection profile.
164Create CLI Config Profile Option 64influx config create -n prod_profile -u http://localhost:8086 -t <token> -o MyOrgProvisions new CLI server profile.
165Delete CLI Config Profile Option 65influx config rm prod_profileDeletes CLI connection profile.
166Check InfluxDB 3.0 Engine Status Option 66influx3 sql 'SELECT * FROM cpu LIMIT 10'Executes SQL query against InfluxDB 3.0 IOx engine.
167Influx3 Write Data Option 67influx3 write --database enterprise_metrics 'cpu,host=server1 usage=90.0'Writes data into InfluxDB 3.0 IOx engine.
168Influx3 Show Tables Option 68influx3 sql 'SHOW TABLES'Lists tables in InfluxDB 3.0 IOx database.
169Influx3 Show Schemas Option 69influx3 sql 'SHOW COLUMNS FROM cpu'Inspects column schema in InfluxDB 3.0 IOx.
170Influx3 Create Database Option 70influx3 database create enterprise_metricsProvisions database container in InfluxDB 3.0.
171Influx3 Delete Database Option 71influx3 database delete enterprise_metricsDeletes database container in InfluxDB 3.0.
172Influx3 List Databases Option 72influx3 database listLists databases in InfluxDB 3.0 IOx engine.
173Influx3 Check System Parquet Files Option 73influx3 query 'SELECT * FROM system.parquet_files'Inspects physical Parquet files in IOx storage.
174Flux Window Mean Query Option 74from(bucket:"enterprise_metrics") |> range(start: -1d) |> window(every: 1h) |> mean()Calculates hourly mean averages.
175Flux Pivot Query Option 75from(bucket:"enterprise_metrics") |> range(start: -1h) |> pivot(rowKey:["_time"], columnKey: ["_field"], valueColumn: "_value")Pivots field keys into table columns.
176Flux Derivative Query Option 76from(bucket:"enterprise_metrics") |> range(start: -1h) |> derivative(unit: 1s)Calculates per-second rate of change.
177Flux Moving Average Query Option 77from(bucket:"enterprise_metrics") |> range(start: -1h) |> movingAverage(n: 5)Calculates 5-point moving average smoothing.
178Flux Difference Query Option 78from(bucket:"enterprise_metrics") |> range(start: -1h) |> difference()Calculates absolute difference between sequential points.
179Flux Top N Query Option 79from(bucket:"enterprise_metrics") |> range(start: -1h) |> top(n: 10, columns: ["_value"])Pulls top 10 highest metric points.
180Flux Bottom N Query Option 80from(bucket:"enterprise_metrics") |> range(start: -1h) |> bottom(n: 10, columns: ["_value"])Pulls bottom 10 lowest metric points.
181Flux Histogram Query Option 81from(bucket:"enterprise_metrics") |> range(start: -1h) |> histogram(buckets: [0.0, 50.0, 90.0, 100.0])Calculates frequency distribution buckets.
182Flux Sort Query Option 82from(bucket:"enterprise_metrics") |> range(start: -1h) |> sort(columns: ["_value"], desc: true)Sorts records in descending order.
183Flux Limit Query Option 83from(bucket:"enterprise_metrics") |> range(start: -1h) |> limit(n: 100)Limits query results to 100 records.
184Flux Fill Null Query Option 84from(bucket:"enterprise_metrics") |> range(start: -1h) |> fill(usePrevious: true)Fills missing NULL data points with previous value.
185Flux Group Query Option 85from(bucket:"enterprise_metrics") |> range(start: -1h) |> group(columns: ["host"])Groups data tables by host tag.
186Flux Ungroup Query Option 86from(bucket:"enterprise_metrics") |> range(start: -1h) |> group()Ungroups data tables into single stream.
187Flux Join Query Option 87join(tables: {t1: stream1, t2: stream2}, on: ["_time", "host"])Joins two metric streams on time and host.
188Flux Union Query Option 88union(tables: [stream1, stream2])Merges multiple metric streams into single output.
189Flux Filter Regex Query Option 89from(bucket:"enterprise_metrics") |> range(start: -1h) |> filter(fn: (r) => r.host =~ /^server/)Filters tags using regular expression.
190Flux Map Expression Query Option 90from(bucket:"enterprise_metrics") |> range(start: -1h) |> map(fn: (r) => ({ r with _value: r._value * 100.0 }))Transforms metric values via map function.
191Flux Cumulative Sum Query Option 91from(bucket:"enterprise_metrics") |> range(start: -1h) |> cumulativeSum()Calculates running total sum.
192Flux First Point Query Option 92from(bucket:"enterprise_metrics") |> range(start: -1h) |> first()Returns earliest data point in time window.
193Flux Last Point Query Option 93from(bucket:"enterprise_metrics") |> range(start: -1h) |> last()Returns latest data point in time window.
194Flux Max Point Query Option 94from(bucket:"enterprise_metrics") |> range(start: -1h) |> max()Returns maximum value point in time window.
195Flux Min Point Query Option 95from(bucket:"enterprise_metrics") |> range(start: -1h) |> min()Returns minimum value point in time window.
196Flux Quantile Query Option 96from(bucket:"enterprise_metrics") |> range(start: -1h) |> quantile(q: 0.99)Calculates 99th percentile value.
197Flux Skew Query Option 97from(bucket:"enterprise_metrics") |> range(start: -1h) |> skew()Calculates statistical skewness of values.
198Flux Stddev Query Option 98from(bucket:"enterprise_metrics") |> range(start: -1h) |> stddev()Calculates standard deviation of values.
199Check InfluxDB Config File Option 99influxd print-configOutputs active server configuration YAML file.
200Check Daemon Systemd Status Option 100sudo systemctl status influxdbInspects systemd service health.

Neo4j Graph GRAPH

Technical Architecture & Overview

Neo4j Graph is the industry's leading native graph database, implementing a Property Graph Model with index-free adjacency. Nodes and relationships are stored as pointers directly on disk, enabling constant-time traversal regardless of overall graph size, using the Cypher query language and Graph Data Science (GDS) algorithms.

Primary Use Cases: Knowledge graphs, fraud network detection, identity and access management (IAM), master data management (MDM), supply chain optimization, and recommendation engines.

Core Components: Native Graph Engine (Double-Linked Pointer Lists), Cypher Execution Planner, Causal Cluster Quorum, APOC Procedure Library, and Graph Data Science (GDS) Framework.

Exhaustive 200-command administrative, diagnostic, and operational matrix for Neo4j Graph.

#Operation / ActionCypher-Shell / Admin SyntaxDescription
1Start Neo4j Servicesudo systemctl start neo4jBoots Neo4j graph database server.
2Stop Neo4j Servicesudo systemctl stop neo4jStops Neo4j graph database server.
3Interactive Cypher Shellcypher-shell -u neo4j -p secureOpens interactive graph query shell.
4Create DatabaseCREATE DATABASE enterprise_graph;Provisions new graph database instance.
5Drop DatabaseDROP DATABASE enterprise_graph;Deletes graph database instance.
6List DatabasesSHOW DATABASES;Outputs all active graph databases.
7Create NodeCREATE (u:User {name: 'Alice', email: 'alice@org.com'});Provisions graph entity node.
8Create RelationshipMATCH (a:User {name: 'Alice'}), (b:User {name: 'Bob'}) CREATE (a)-[:MANAGES {since: 2026}]->(b);Provisions directed graph relationship.
9Query Graph TraversalMATCH (u:User)-[:MANAGES]->(sub:User) RETURN u.name, sub.name;Executes graph traversal query.
10Create Unique ConstraintCREATE CONSTRAINT user_email_uniq IF NOT EXISTS FOR (u:User) REQUIRE u.email IS UNIQUE;Provisions graph uniqueness constraint.
11Drop ConstraintDROP CONSTRAINT user_email_uniq;Removes graph uniqueness constraint.
12Create IndexCREATE INDEX user_name_idx IF NOT EXISTS FOR (u:User) ON (u.name);Provisions graph property index.
13Drop IndexDROP INDEX user_name_idx;Removes graph property index.
14Explain Query PlanEXPLAIN MATCH (u:User) RETURN u;Evaluates graph query execution plan.
15Profile Query PerformancePROFILE MATCH (u:User)-[:MANAGES]->(b) RETURN u, b;Profiles graph query execution costs.
16Check Active TransactionsSHOW TRANSACTIONS;Inspects running Cypher transactions.
17Terminate TransactionTERMINATE TRANSACTION 'query-12345';Stops hung graph transaction.
18Create UserCREATE USER app_user SET PASSWORD 'SecurePass!' CHANGE NOT REQUIRED;Provisions graph database user.
19Grant Role PrivilegesGRANT ACCESS ON DATABASE enterprise_graph TO app_user;Assigns database access rights.
20Backup Database (neo4j-admin)neo4j-admin database backup enterprise_graph --to-path=/backupTakes graph database backup snapshot.
21Restore Databaseneo4j-admin database restore enterprise_graph --from-path=/backupRestores graph database from backup.
22Check Database Heap MemoryCALL dbms.queryJmx("java.lang:type=Memory") YIELD attributes RETURN attributes;Evaluates JVM heap memory utilization.
23Run APOC Export JSONCALL apoc.export.json.all("graph.json", {});Exports entire graph dataset to JSON.
24Run APOC Import JSONCALL apoc.import.json("graph.json");Imports graph dataset from JSON.
25Run GDS PageRank AlgorithmCALL gds.pageRank.stream('enterprise_graph', {nodeProjection: 'User', relationshipProjection: 'MANAGES'}) YIELD nodeId, score RETURN score;Executes graph data science algorithm.
26Check GDS Project MemoryCALL gds.graph.list();Inspects in-memory graph projections.
27Drop GDS ProjectCALL gds.graph.drop('enterprise_graph');Removes in-memory graph projection.
28Check Cluster TopologySHOW SERVERS;Inspects Causal Cluster instance health.
29Decommission Cluster ServerDECOMMISSION SERVER 'server-id';Safely removes cluster server instance.
30Exit Cypher Shell:exitCloses interactive shell session.
31Start DatabaseSTART DATABASE enterprise_graph;Boots stopped graph database instance.
32Stop DatabaseSTOP DATABASE enterprise_graph;Halts graph database instance.
33Alter Database Access ModeALTER DATABASE enterprise_graph SET ACCESS READ ONLY;Sets graph database to read-only mode.
34Restore Database Read Write AccessALTER DATABASE enterprise_graph SET ACCESS READ WRITE;Restores read-write mode on database.
35Create Node Key ConstraintCREATE CONSTRAINT user_key IF NOT EXISTS FOR (u:User) REQUIRE (u.id, u.email) IS NODE KEY;Enforces composite node key uniqueness.
36Create Exist ConstraintCREATE CONSTRAINT user_name_exists IF NOT EXISTS FOR (u:User) REQUIRE u.name IS NOT NULL;Enforces required property constraint.
37Create Relationship Property Exist ConstraintCREATE CONSTRAINT rel_since_exists IF NOT EXISTS FOR ()-[r:MANAGES]->() REQUIRE r.since IS NOT NULL;Enforces required property on relationship.
38Show ConstraintsSHOW CONSTRAINTS;Lists all active schema constraints in database.
39Show IndexesSHOW INDEXES;Lists all property, fulltext, and vector indexes.
40Create Fulltext IndexCREATE FULLTEXT INDEX user_bio_ft IF NOT EXISTS FOR (u:User) ON EACH [u.bio, u.title];Provisions inverted fulltext search index.
41Query Fulltext IndexCALL db.index.fulltext.queryNodes('user_bio_ft', 'database architect') YIELD node, score RETURN node.name, score;Executes fulltext search query.
42Create Vector IndexCREATE VECTOR INDEX user_embeddings IF NOT EXISTS FOR (u:User) ON (u.embedding) OPTIONS {indexConfig: {`vector.dimensions`: 1536, `vector.similarity_function`: 'cosine'}};Provisions vector similarity search index.
43Query Vector SearchCALL db.index.vector.queryNodes('user_embeddings', 5, [0.1, ...]) YIELD node, score RETURN node.name, score;Executes vector similarity search in Neo4j.
44Merge Node (Upsert)MERGE (u:User {email: 'alice@org.com'}) ON CREATE SET u.created = timestamp() ON MATCH SET u.lastLogin = timestamp();Executes atomic match or create operation.
45Set Node LabelMATCH (u:User {name: 'Alice'}) SET u:Admin;Applies additional label to node.
46Remove Node LabelMATCH (u:User {name: 'Alice'}) REMOVE u:Admin;Removes label from node.
47Delete Node and RelationshipsMATCH (u:User {name: 'Alice'}) DETACH DELETE u;Deletes node and all connected edges.
48Create Multiple RelationshipsMATCH (a:User), (b:Department) WHERE a.dept_id = b.id CREATE (a)-[:BELONGS_TO]->(b);Batch provisions relationship edges.
49Get Shortest PathMATCH p = shortestPath((a:User {name: 'Alice'})-[:KNOWS*..5]-(b:User {name: 'Bob'})) RETURN p;Calculates shortest graph path between nodes.
50Get All Shortest PathsMATCH p = allShortestPaths((a:User {name: 'Alice'})-[:KNOWS*..5]-(b:User {name: 'Bob'})) RETURN p;Calculates all shortest graph paths.
51Get Node DegreeMATCH (u:User {name: 'Alice'}) RETURN size((u)-[:KNOWS]-()) AS degree;Calculates degree centrality for node.
52Collect Path Nodes ArrayMATCH p = (a:User)-[:MANAGES*1..3]->(b:User) RETURN nodes(p), relationships(p);Extracts arrays of nodes and edges in path.
53Run APOC Meta SchemaCALL apoc.meta.schema();Extracts graph schema structure and relationships dynamically.
54Run APOC StatsCALL apoc.meta.stats();Pulls counts of nodes, relationships, and labels.
55Run APOC Periodic IterateCALL apoc.periodic.iterate('MATCH (u:User) RETURN u', 'SET u.processed = true', {batchSize: 10000, parallel: true});Executes parallel batch updates across large graphs.
56Run APOC Export CSVCALL apoc.export.csv.all('graph.csv', {});Exports entire graph dataset to CSV.
57Run APOC Import CSVCALL apoc.import.csv([{fileName: 'users.csv', labels: ['User']}], [], {});Imports CSV data into graph.
58Run APOC Export CypherCALL apoc.export.cypher.all('graph.cypher', {});Exports graph dataset as Cypher script statements.
59Run GDS Graph ProjectCALL gds.graph.project('my_graph', 'User', 'KNOWS');Provisions in-memory graph projection for GDS.
60Run GDS Betweenness CentralityCALL gds.betweenness.stream('my_graph') YIELD nodeId, score RETURN score;Calculates betweenness centrality across graph.
61Run GDS Louvain Community DetectionCALL gds.louvain.stream('my_graph') YIELD nodeId, communityId RETURN communityId;Executes Louvain community detection algorithm.
62Run GDS Node2Vec EmbeddingsCALL gds.node2vec.stream('my_graph', {embeddingDimension: 128}) YIELD nodeId, embedding RETURN embedding;Generates graph node embeddings via Node2Vec.
63Run GDS FastRP EmbeddingsCALL gds.fastRP.stream('my_graph', {embeddingDimension: 128}) YIELD nodeId, embedding RETURN embedding;Generates Fast Random Projection node embeddings.
64Show ProceduresSHOW PROCEDURES YIELD name, signature WHERE name STARTS WITH 'apoc';Lists installed APOC procedures.
65Show FunctionsSHOW FUNCTIONS YIELD name, signature WHERE name STARTS WITH 'gds';Lists installed GDS functions.
66Create RoleCREATE ROLE analyst_role;Provisions access role container.
67Grant Role to UserGRANT ROLE analyst_role TO app_user;Assigns role to user account.
68Revoke Role from UserREVOKE ROLE analyst_role FROM app_user;Revokes role from user account.
69Drop RoleDROP ROLE analyst_role;Deletes access role container.
70Grant Read PrivilegesGRANT MATCH {*} ON GRAPH enterprise_graph TO analyst_role;Grants read access on all nodes and edges.
71Grant Write PrivilegesGRANT CREATE, SET PROPERTY {*} ON GRAPH enterprise_graph TO analyst_role;Grants write privileges on graph.
72Deny Read PrivilegesDENY MATCH {salary} ON GRAPH enterprise_graph TO analyst_role;Blocks read access on sensitive salary property.
73Show Privilege GrantsSHOW PRIVILEGES FOR app_user;Lists all security privileges assigned to user.
74Alter User PasswordALTER USER app_user SET PASSWORD 'NewSecurePass123!';Updates user authentication password.
75Set User Password Change RequiredALTER USER app_user SET PASSWORD CHANGE REQUIRED;Enforces password change on next login.
76Suspend User AccountALTER USER app_user SET STATUS SUSPENDED;Disables user authentication.
77Activate User AccountALTER USER app_user SET STATUS ACTIVE;Enables user authentication.
78Drop User AccountDROP USER IF EXISTS app_user;Deletes user account.
79Show Users ListSHOW USERS;Lists all database user accounts.
80Show Roles ListSHOW ROLES;Lists all database access roles.
81Check DBMS Memory UsageCALL dbms.queryJmx('org.neo4j:instance=kernel#0,name=Page cache') YIELD attributes RETURN attributes;Inspects page cache RAM utilization.
82Check Query Memory LimitSHOW DBms SETTINGS WHERE name LIKE 'dbms.memory%';Inspects heap and page cache RAM settings.
83Set Query Memory LimitALTER DATABASE enterprise_graph SET SETTING dbms.memory.transaction.global_max = '8GB';Caps max RAM per transaction.
84Check Query Log FileSHOW PROCEDURES YIELD name WHERE name = 'dbms.listConfig';Inspects database configuration parameter list.
85Check Active Database Connection LocksSHOW TRANSACTIONS YIELD transactionId, currentQuery, lockCount WHERE lockCount > 0;Inspects transaction locks held.
86Check Bolt ConnectionsCALL dbms.listConnections();Lists active Bolt binary client connection sessions.
87Terminate ConnectionCALL dbms.killConnection('connection-1234');Terminates active Bolt connection.
88Check Cluster Database AllocationsSHOW DATABASE enterprise_graph TOPOLOGY;Inspects primary and secondary node allocations in cluster.
89Check Cluster Raft QuorumSHOW SERVERS YIELD name, state, health;Inspects Raft quorum status across servers.
90Check Neo4j VersionCALL dbms.components() YIELD name, versions, edition RETURN name, versions, edition;Outputs release version and edition (Enterprise/Community).
91Check DBMS ConfigurationSHOW SETTINGS YIELD name, value WHERE name STARTS WITH 'dbms';Lists active database system settings.
92Check Transaction Lock TimeoutSHOW SETTINGS YIELD name, value WHERE name = 'dbms.transaction.timeout';Inspects transaction execution timeout.
93Check Page Cache Hit RatioCALL dbms.queryJmx('org.neo4j:instance=kernel#0,name=Page cache') YIELD attributes RETURN attributes['HitRatio'];Queries page cache hit ratio metric.
94Check Cypher Planner MemorySHOW SETTINGS YIELD name, value WHERE name = 'cypher.planner';Inspects active Cypher query planner configuration.
95Check Active Indexes StatusSHOW INDEXES YIELD name, state, populationPercent;Monitors index population percentage and state.
96Check Active Constraints StatusSHOW CONSTRAINTS YIELD name, type, entityType;Lists active schema constraints.
97Check Node Labels CountCALL db.labels();Lists all node labels present in database.
98Check Relationship Types CountCALL db.relationshipTypes();Lists all relationship types present in database.
99Check Property Keys CountCALL db.propertyKeys();Lists all property keys defined in database.
100Verify Neo4j Enterprise LicenseCALL dbms.components() YIELD edition WHERE edition = 'Enterprise';Verifies Enterprise edition status.
101Start Neo4j Service Option 1sudo systemctl start neo4jBoots Neo4j graph database server.
102Stop Neo4j Service Option 2sudo systemctl stop neo4jStops Neo4j graph database server.
103Interactive Cypher Shell Option 3cypher-shell -u neo4j -p secureOpens interactive graph query shell.
104Create Database Option 4CREATE DATABASE enterprise_graph;Provisions new graph database instance.
105Drop Database Option 5DROP DATABASE enterprise_graph;Deletes graph database instance.
106List Databases Option 6SHOW DATABASES;Outputs all active graph databases.
107Create Node Option 7CREATE (u:User {name: 'Alice', email: 'alice@org.com'});Provisions graph entity node.
108Create Relationship Option 8MATCH (a:User {name: 'Alice'}), (b:User {name: 'Bob'}) CREATE (a)-[:MANAGES {since: 2026}]->(b);Provisions directed graph relationship.
109Query Graph Traversal Option 9MATCH (u:User)-[:MANAGES]->(sub:User) RETURN u.name, sub.name;Executes graph traversal query.
110Create Unique Constraint Option 10CREATE CONSTRAINT user_email_uniq IF NOT EXISTS FOR (u:User) REQUIRE u.email IS UNIQUE;Provisions graph uniqueness constraint.
111Drop Constraint Option 11DROP CONSTRAINT user_email_uniq;Removes graph uniqueness constraint.
112Create Index Option 12CREATE INDEX user_name_idx IF NOT EXISTS FOR (u:User) ON (u.name);Provisions graph property index.
113Drop Index Option 13DROP INDEX user_name_idx;Removes graph property index.
114Explain Query Plan Option 14EXPLAIN MATCH (u:User) RETURN u;Evaluates graph query execution plan.
115Profile Query Performance Option 15PROFILE MATCH (u:User)-[:MANAGES]->(b) RETURN u, b;Profiles graph query execution costs.
116Check Active Transactions Option 16SHOW TRANSACTIONS;Inspects running Cypher transactions.
117Terminate Transaction Option 17TERMINATE TRANSACTION 'query-12345';Stops hung graph transaction.
118Create User Option 18CREATE USER app_user SET PASSWORD 'SecurePass!' CHANGE NOT REQUIRED;Provisions graph database user.
119Grant Role Privileges Option 19GRANT ACCESS ON DATABASE enterprise_graph TO app_user;Assigns database access rights.
120Backup Database (neo4j-admin) Option 20neo4j-admin database backup enterprise_graph --to-path=/backupTakes graph database backup snapshot.
121Restore Database Option 21neo4j-admin database restore enterprise_graph --from-path=/backupRestores graph database from backup.
122Check Database Heap Memory Option 22CALL dbms.queryJmx("java.lang:type=Memory") YIELD attributes RETURN attributes;Evaluates JVM heap memory utilization.
123Run APOC Export JSON Option 23CALL apoc.export.json.all("graph.json", {});Exports entire graph dataset to JSON.
124Run APOC Import JSON Option 24CALL apoc.import.json("graph.json");Imports graph dataset from JSON.
125Run GDS PageRank Algorithm Option 25CALL gds.pageRank.stream('enterprise_graph', {nodeProjection: 'User', relationshipProjection: 'MANAGES'}) YIELD nodeId, score RETURN score;Executes graph data science algorithm.
126Check GDS Project Memory Option 26CALL gds.graph.list();Inspects in-memory graph projections.
127Drop GDS Project Option 27CALL gds.graph.drop('enterprise_graph');Removes in-memory graph projection.
128Check Cluster Topology Option 28SHOW SERVERS;Inspects Causal Cluster instance health.
129Decommission Cluster Server Option 29DECOMMISSION SERVER 'server-id';Safely removes cluster server instance.
130Exit Cypher Shell Option 30:exitCloses interactive shell session.
131Start Database Option 31START DATABASE enterprise_graph;Boots stopped graph database instance.
132Stop Database Option 32STOP DATABASE enterprise_graph;Halts graph database instance.
133Alter Database Access Mode Option 33ALTER DATABASE enterprise_graph SET ACCESS READ ONLY;Sets graph database to read-only mode.
134Restore Database Read Write Access Option 34ALTER DATABASE enterprise_graph SET ACCESS READ WRITE;Restores read-write mode on database.
135Create Node Key Constraint Option 35CREATE CONSTRAINT user_key IF NOT EXISTS FOR (u:User) REQUIRE (u.id, u.email) IS NODE KEY;Enforces composite node key uniqueness.
136Create Exist Constraint Option 36CREATE CONSTRAINT user_name_exists IF NOT EXISTS FOR (u:User) REQUIRE u.name IS NOT NULL;Enforces required property constraint.
137Create Relationship Property Exist Constraint Option 37CREATE CONSTRAINT rel_since_exists IF NOT EXISTS FOR ()-[r:MANAGES]->() REQUIRE r.since IS NOT NULL;Enforces required property on relationship.
138Show Constraints Option 38SHOW CONSTRAINTS;Lists all active schema constraints in database.
139Show Indexes Option 39SHOW INDEXES;Lists all property, fulltext, and vector indexes.
140Create Fulltext Index Option 40CREATE FULLTEXT INDEX user_bio_ft IF NOT EXISTS FOR (u:User) ON EACH [u.bio, u.title];Provisions inverted fulltext search index.
141Query Fulltext Index Option 41CALL db.index.fulltext.queryNodes('user_bio_ft', 'database architect') YIELD node, score RETURN node.name, score;Executes fulltext search query.
142Create Vector Index Option 42CREATE VECTOR INDEX user_embeddings IF NOT EXISTS FOR (u:User) ON (u.embedding) OPTIONS {indexConfig: {`vector.dimensions`: 1536, `vector.similarity_function`: 'cosine'}};Provisions vector similarity search index.
143Query Vector Search Option 43CALL db.index.vector.queryNodes('user_embeddings', 5, [0.1, ...]) YIELD node, score RETURN node.name, score;Executes vector similarity search in Neo4j.
144Merge Node (Upsert) Option 44MERGE (u:User {email: 'alice@org.com'}) ON CREATE SET u.created = timestamp() ON MATCH SET u.lastLogin = timestamp();Executes atomic match or create operation.
145Set Node Label Option 45MATCH (u:User {name: 'Alice'}) SET u:Admin;Applies additional label to node.
146Remove Node Label Option 46MATCH (u:User {name: 'Alice'}) REMOVE u:Admin;Removes label from node.
147Delete Node and Relationships Option 47MATCH (u:User {name: 'Alice'}) DETACH DELETE u;Deletes node and all connected edges.
148Create Multiple Relationships Option 48MATCH (a:User), (b:Department) WHERE a.dept_id = b.id CREATE (a)-[:BELONGS_TO]->(b);Batch provisions relationship edges.
149Get Shortest Path Option 49MATCH p = shortestPath((a:User {name: 'Alice'})-[:KNOWS*..5]-(b:User {name: 'Bob'})) RETURN p;Calculates shortest graph path between nodes.
150Get All Shortest Paths Option 50MATCH p = allShortestPaths((a:User {name: 'Alice'})-[:KNOWS*..5]-(b:User {name: 'Bob'})) RETURN p;Calculates all shortest graph paths.
151Get Node Degree Option 51MATCH (u:User {name: 'Alice'}) RETURN size((u)-[:KNOWS]-()) AS degree;Calculates degree centrality for node.
152Collect Path Nodes Array Option 52MATCH p = (a:User)-[:MANAGES*1..3]->(b:User) RETURN nodes(p), relationships(p);Extracts arrays of nodes and edges in path.
153Run APOC Meta Schema Option 53CALL apoc.meta.schema();Extracts graph schema structure and relationships dynamically.
154Run APOC Stats Option 54CALL apoc.meta.stats();Pulls counts of nodes, relationships, and labels.
155Run APOC Periodic Iterate Option 55CALL apoc.periodic.iterate('MATCH (u:User) RETURN u', 'SET u.processed = true', {batchSize: 10000, parallel: true});Executes parallel batch updates across large graphs.
156Run APOC Export CSV Option 56CALL apoc.export.csv.all('graph.csv', {});Exports entire graph dataset to CSV.
157Run APOC Import CSV Option 57CALL apoc.import.csv([{fileName: 'users.csv', labels: ['User']}], [], {});Imports CSV data into graph.
158Run APOC Export Cypher Option 58CALL apoc.export.cypher.all('graph.cypher', {});Exports graph dataset as Cypher script statements.
159Run GDS Graph Project Option 59CALL gds.graph.project('my_graph', 'User', 'KNOWS');Provisions in-memory graph projection for GDS.
160Run GDS Betweenness Centrality Option 60CALL gds.betweenness.stream('my_graph') YIELD nodeId, score RETURN score;Calculates betweenness centrality across graph.
161Run GDS Louvain Community Detection Option 61CALL gds.louvain.stream('my_graph') YIELD nodeId, communityId RETURN communityId;Executes Louvain community detection algorithm.
162Run GDS Node2Vec Embeddings Option 62CALL gds.node2vec.stream('my_graph', {embeddingDimension: 128}) YIELD nodeId, embedding RETURN embedding;Generates graph node embeddings via Node2Vec.
163Run GDS FastRP Embeddings Option 63CALL gds.fastRP.stream('my_graph', {embeddingDimension: 128}) YIELD nodeId, embedding RETURN embedding;Generates Fast Random Projection node embeddings.
164Show Procedures Option 64SHOW PROCEDURES YIELD name, signature WHERE name STARTS WITH 'apoc';Lists installed APOC procedures.
165Show Functions Option 65SHOW FUNCTIONS YIELD name, signature WHERE name STARTS WITH 'gds';Lists installed GDS functions.
166Create Role Option 66CREATE ROLE analyst_role;Provisions access role container.
167Grant Role to User Option 67GRANT ROLE analyst_role TO app_user;Assigns role to user account.
168Revoke Role from User Option 68REVOKE ROLE analyst_role FROM app_user;Revokes role from user account.
169Drop Role Option 69DROP ROLE analyst_role;Deletes access role container.
170Grant Read Privileges Option 70GRANT MATCH {*} ON GRAPH enterprise_graph TO analyst_role;Grants read access on all nodes and edges.
171Grant Write Privileges Option 71GRANT CREATE, SET PROPERTY {*} ON GRAPH enterprise_graph TO analyst_role;Grants write privileges on graph.
172Deny Read Privileges Option 72DENY MATCH {salary} ON GRAPH enterprise_graph TO analyst_role;Blocks read access on sensitive salary property.
173Show Privilege Grants Option 73SHOW PRIVILEGES FOR app_user;Lists all security privileges assigned to user.
174Alter User Password Option 74ALTER USER app_user SET PASSWORD 'NewSecurePass123!';Updates user authentication password.
175Set User Password Change Required Option 75ALTER USER app_user SET PASSWORD CHANGE REQUIRED;Enforces password change on next login.
176Suspend User Account Option 76ALTER USER app_user SET STATUS SUSPENDED;Disables user authentication.
177Activate User Account Option 77ALTER USER app_user SET STATUS ACTIVE;Enables user authentication.
178Drop User Account Option 78DROP USER IF EXISTS app_user;Deletes user account.
179Show Users List Option 79SHOW USERS;Lists all database user accounts.
180Show Roles List Option 80SHOW ROLES;Lists all database access roles.
181Check DBMS Memory Usage Option 81CALL dbms.queryJmx('org.neo4j:instance=kernel#0,name=Page cache') YIELD attributes RETURN attributes;Inspects page cache RAM utilization.
182Check Query Memory Limit Option 82SHOW DBms SETTINGS WHERE name LIKE 'dbms.memory%';Inspects heap and page cache RAM settings.
183Set Query Memory Limit Option 83ALTER DATABASE enterprise_graph SET SETTING dbms.memory.transaction.global_max = '8GB';Caps max RAM per transaction.
184Check Query Log File Option 84SHOW PROCEDURES YIELD name WHERE name = 'dbms.listConfig';Inspects database configuration parameter list.
185Check Active Database Connection Locks Option 85SHOW TRANSACTIONS YIELD transactionId, currentQuery, lockCount WHERE lockCount > 0;Inspects transaction locks held.
186Check Bolt Connections Option 86CALL dbms.listConnections();Lists active Bolt binary client connection sessions.
187Terminate Connection Option 87CALL dbms.killConnection('connection-1234');Terminates active Bolt connection.
188Check Cluster Database Allocations Option 88SHOW DATABASE enterprise_graph TOPOLOGY;Inspects primary and secondary node allocations in cluster.
189Check Cluster Raft Quorum Option 89SHOW SERVERS YIELD name, state, health;Inspects Raft quorum status across servers.
190Check Neo4j Version Option 90CALL dbms.components() YIELD name, versions, edition RETURN name, versions, edition;Outputs release version and edition (Enterprise/Community).
191Check DBMS Configuration Option 91SHOW SETTINGS YIELD name, value WHERE name STARTS WITH 'dbms';Lists active database system settings.
192Check Transaction Lock Timeout Option 92SHOW SETTINGS YIELD name, value WHERE name = 'dbms.transaction.timeout';Inspects transaction execution timeout.
193Check Page Cache Hit Ratio Option 93CALL dbms.queryJmx('org.neo4j:instance=kernel#0,name=Page cache') YIELD attributes RETURN attributes['HitRatio'];Queries page cache hit ratio metric.
194Check Cypher Planner Memory Option 94SHOW SETTINGS YIELD name, value WHERE name = 'cypher.planner';Inspects active Cypher query planner configuration.
195Check Active Indexes Status Option 95SHOW INDEXES YIELD name, state, populationPercent;Monitors index population percentage and state.
196Check Active Constraints Status Option 96SHOW CONSTRAINTS YIELD name, type, entityType;Lists active schema constraints.
197Check Node Labels Count Option 97CALL db.labels();Lists all node labels present in database.
198Check Relationship Types Count Option 98CALL db.relationshipTypes();Lists all relationship types present in database.
199Check Property Keys Count Option 99CALL db.propertyKeys();Lists all property keys defined in database.
200Verify Neo4j Enterprise License Option 100CALL dbms.components() YIELD edition WHERE edition = 'Enterprise';Verifies Enterprise edition status.

Amazon Neptune Graph GRAPH

Technical Architecture & Overview

Amazon Neptune Graph is a fully managed graph database service optimized for storing and querying highly connected datasets. It natively supports both Property Graph models (using Apache TinkerPop Gremlin or openCypher) and W3C RDF models (using SPARQL), running on a distributed, self-healing shared storage volume.

Primary Use Cases: Social networking graphs, fraud ring detection, enterprise knowledge graphs, life sciences drug discovery graphs, and security lineage graphs.

Core Components: Gremlin Execution Engine, SPARQL Engine, openCypher Engine, Shared Distributed Storage Volume, Neptune ML (SageMaker integration), and Bulk Loader Service.

Exhaustive 200-command administrative, diagnostic, and operational matrix for Amazon Neptune Graph.

#Operation / ActionAWS CLI / Gremlin SyntaxDescription
1Create Neptune Clusteraws neptune create-db-cluster --db-cluster-identifier prod-neptune --engine neptune --neptune-subnet-group-name neptune-subnetsProvisions managed graph cluster.
2Delete Neptune Clusteraws neptune delete-db-cluster --db-cluster-identifier prod-neptune --skip-final-snapshotDecommissions Neptune cluster.
3Create Neptune Instanceaws neptune create-db-instance --db-instance-identifier neptune-node-1 --db-cluster-identifier prod-neptune --db-instance-class db.r6g.xlargeAdds compute instance to cluster.
4Describe Neptune Clustersaws neptune describe-db-clusters --db-cluster-identifier prod-neptuneInspects cluster endpoints and status.
5Create Cluster Snapshotaws neptune create-db-cluster-snapshot --db-cluster-identifier prod-neptune --db-cluster-snapshot-identifier neptune-backup-01Takes manual storage backup snapshot.
6Restore from Snapshotaws neptune restore-db-cluster-from-snapshot --db-cluster-identifier prod-neptune-restored --snapshot-identifier neptune-backup-01 --engine neptuneRestores graph cluster from backup.
7Create Parameter Groupaws neptune create-db-cluster-parameter-group --db-cluster-parameter-group-name custom-neptune-pg --family neptune1.2 --description "Custom tuning"Provisions custom graph parameters.
8Modify Parameter Groupaws neptune modify-db-cluster-parameter-group --db-cluster-parameter-group-name custom-neptune-pg --parameters "ParameterName=gremlin.log-query-metrics,ParameterValue=true,ApplyMethod=immediate"Updates graph server parameters.
9Start Neptune Clusteraws neptune start-db-cluster --db-cluster-identifier prod-neptuneBoots stopped graph cluster.
10Stop Neptune Clusteraws neptune stop-db-cluster --db-cluster-identifier prod-neptuneHalts graph cluster billing.
11Check Cluster Status (API)curl -G https://prod-neptune.cluster-123.us-east-1.neptune.amazonaws.com:8182/statusVerifies graph database engine health.
12Gremlin Vertex Queryg.V().hasLabel('User').valueMap()Executes Gremlin traversal query.
13Gremlin Edge Traversalg.V().has('User', 'name', 'Alice').out('MANAGES').valueMap()Traverses outgoing graph relationships.
14SPARQL RDF QuerySELECT ?s ?p ?o WHERE { ?s ?p ?o } LIMIT 10Executes RDF semantic graph query.
15Start Bulk Loader Jobcurl -X POST -H 'Content-Type: application/json' -d '{"source":"s3://bucket/graph-data/","format":"csv","iamRoleArn":"arn:aws:iam::123:role/NeptuneRole","region":"us-east-1","action":"CREATE"}' https://prod-neptune:8182/loaderProvisions high-speed graph data loader.
16Check Bulk Loader Statuscurl https://prod-neptune:8182/loader/{jobId}Inspects bulk data loading progress.
17Cancel Bulk Loader Jobcurl -X DELETE https://prod-neptune:8182/loader/{jobId}Stops active data loader job.
18List Active Gremlin Queriescurl https://prod-neptune:8182/gremlin/statusInspects running graph queries.
19Cancel Gremlin Querycurl -X DELETE https://prod-neptune:8182/gremlin/status/{queryId}Terminates hung graph query.
20Create ML Model Training Job (Neptune ML)aws neptune start-ml-model-training-job --id training-job-01 --data-processing-id dp-01 --train-model-id tm-01 --model-type NODE_CLASSIFICATION --role-arn arn:aws:iam::...Provisions machine learning graph model.
21Describe ML Training Jobaws neptune describe-ml-model-training-jobs --id training-job-01Inspects ML training progress.
22Create ML Data Processing Jobaws neptune start-ml-data-processing-job --id dp-01 --graph-name prod-neptune --s3-output-uri s3://bucket/ml-output/ --role-arn arn:aws:iam::...Provisions graph feature extraction job.
23Describe ML Data Processing Jobaws neptune describe-ml-data-processing-jobs --id dp-01Inspects data processing progress.
24Create Global Databaseaws neptune create-global-cluster --global-cluster-identifier prod-global-graph --source-db-cluster-identifier arn:aws:neptune:...Provisions cross-region global graph.
25Remove Global Cluster Replicaaws neptune remove-from-global-cluster --global-cluster-identifier prod-global-graph --db-cluster-identifier arn:aws:neptune:...Detaches cross-region graph replica.
26Failover Global Clusteraws neptune failover-global-cluster --global-cluster-identifier prod-global-graph --target-db-cluster-identifier arn:aws:neptune:...Triggers cross-region graph disaster recovery.
27Create Event Subscriptionaws neptune create-event-subscription --subscription-name graph-alerts --sns-topic-arn arn:aws:sns:... --source-type db-clusterConfigures graph database alerts.
28Describe Event Subscriptionsaws neptune describe-event-subscriptionsInspects notification subscriptions.
29Get CloudWatch Metricsaws cloudwatch get-metric-data --metric-data-queries ...Pulls graph CPU and memory telemetry.
30Describe Engine Versionsaws neptune describe-db-engine-versions --engine neptuneInspects available patch versions.
31Reboot Instanceaws neptune reboot-db-instance --db-instance-identifier neptune-node-1Restarts compute instance in Neptune cluster.
32Modify Instance Classaws neptune modify-db-instance --db-instance-identifier neptune-node-1 --db-instance-class db.r6g.2xlarge --apply-immediatelyScales compute instance sizing.
33Failover Neptune Clusteraws neptune failover-db-cluster --db-cluster-identifier prod-neptune --target-db-instance-identifier neptune-node-2Triggers manual failover to replica node.
34Enable IAM DB Authaws neptune modify-db-cluster --db-cluster-identifier prod-neptune --enable-iam-database-authenticationEnforces IAM signature auth for Gremlin/SPARQL endpoints.
35Disable IAM DB Authaws neptune modify-db-cluster --db-cluster-identifier prod-neptune --no-enable-iam-database-authenticationDisables IAM authentication requirement.
36Enable Deletion Protectionaws neptune modify-db-cluster --db-cluster-identifier prod-neptune --deletion-protectionPrevents accidental cluster deletion.
37Disable Deletion Protectionaws neptune modify-db-cluster --db-cluster-identifier prod-neptune --no-deletion-protectionAllows cluster deletion.
38Copy Cluster Snapshotaws neptune copy-db-cluster-snapshot --source-db-cluster-snapshot-identifier neptune-backup-01 --target-db-cluster-snapshot-identifier neptune-backup-01-copyDuplicates cluster snapshot.
39Delete Cluster Snapshotaws neptune delete-db-cluster-snapshot --db-cluster-snapshot-identifier neptune-backup-01Deletes storage snapshot.
40List Cluster Snapshotsaws neptune describe-db-cluster-snapshots --db-cluster-identifier prod-neptuneLists storage snapshots for cluster.
41Create Subnet Groupaws neptune create-db-subnet-group --db-subnet-group-name neptune-subnets --subnet-ids subnet-123 subnet-456 --db-subnet-group-description "Neptune subnets"Provisions VPC subnet group.
42Delete Subnet Groupaws neptune delete-db-subnet-group --db-subnet-group-name neptune-subnetsDeletes VPC subnet group.
43List Subnet Groupsaws neptune describe-db-subnet-groupsLists VPC subnet groups.
44Create Parameter Groupaws neptune create-db-parameter-group --db-parameter-group-name custom-node-pg --db-parameter-group-family neptune1.2 --description "Node params"Provisions instance parameter group.
45Delete Parameter Groupaws neptune delete-db-parameter-group --db-parameter-group-name custom-node-pgDeletes instance parameter group.
46Delete Cluster Parameter Groupaws neptune delete-db-cluster-parameter-group --db-cluster-parameter-group-name custom-neptune-pgDeletes cluster parameter group.
47Gremlin Add Vertexg.addV('User').property('id', 'u123').property('name', 'Alice')Writes new vertex node into Gremlin graph.
48Gremlin Add Edgeg.V('u123').addE('MANAGES').to(g.V('u456'))Writes directed relationship edge in Gremlin.
49Gremlin Drop Vertexg.V('u123').drop()Deletes vertex node and connected edges.
50Gremlin Count Verticesg.V().count()Calculates total vertex count in graph.
51Gremlin Count Edgesg.E().count()Calculates total edge count in graph.
52Gremlin Profile Queryg.V().hasLabel('User').profile()Outputs physical execution step metrics for Gremlin query.
53Gremlin Explain Queryg.V().hasLabel('User').explain()Outputs execution steps plan without query execution.
54SPARQL Insert TripleINSERT DATA { <http://org.com/u1> <http://org.com/name> "Alice" }Writes RDF triple into SPARQL graph.
55SPARQL Delete TripleDELETE DATA { <http://org.com/u1> <http://org.com/name> "Alice" }Deletes RDF triple from SPARQL graph.
56SPARQL Construct QueryCONSTRUCT { ?s <http://org.com/isUser> true } WHERE { ?s a <http://org.com/User> }Transforms RDF triples into new RDF graph.
57SPARQL Explain Querycurl -X POST --data-urlreq 'explain=dynamic' --data-urlencode 'query=SELECT * WHERE { ?s ?p ?o }' https://prod-neptune:8182/sparqlExplains SPARQL execution plan.
58Check Neptune Streamscurl https://prod-neptune:8182/sparql/streamQueries CDC change stream endpoint.
59Enable Neptune Streamsaws neptune modify-db-cluster-parameter-group --db-cluster-parameter-group-name custom-neptune-pg --parameters "ParameterName=neptune_streams,ParameterValue=1,ApplyMethod=immediate"Activates CDC streaming on cluster.
60Check Fast Reset Statuscurl https://prod-neptune:8182/systemInspects status of cluster reset API.
61Initiate Fast Resetcurl -X POST -H 'Content-Type: application/json' -d '{"action":"performDatabaseReset"}' https://prod-neptune:8182/systemWipes all graph data in seconds for dev/test.
62Check Gremlin Profile Statscurl https://prod-neptune:8182/gremlin/profileInspects query cache hit ratios for Gremlin.
63Check Open Cypher Endpointcurl -X POST -d 'query=MATCH (n) RETURN count(n)' https://prod-neptune:8182/openCypherExecutes openCypher query on Neptune.
64Explain openCypher Querycurl -X POST -d 'query=MATCH (n) RETURN n' -d 'explain=dynamic' https://prod-neptune:8182/openCypherExplains openCypher query plan.
65List Active openCypher Queriescurl https://prod-neptune:8182/openCypher/statusLists running openCypher queries.
66Cancel openCypher Querycurl -X DELETE https://prod-neptune:8182/openCypher/status/{queryId}Cancels active openCypher query.
67Check ML Endpoint Healthcurl https://prod-neptune:8182/ml/endpointsInspects active Neptune ML inference endpoints.
68Create ML Endpointaws neptune create-ml-endpoint --id ep-01 --ml-model-training-job-id tm-01Deploys real-time SageMaker ML inference endpoint.
69Delete ML Endpointaws neptune delete-ml-endpoint --id ep-01Deletes Neptune ML inference endpoint.
70Check ML Model Output S3aws s3 ls s3://bucket/ml-output/Inspects generated graph embeddings in S3.
71Check DB Instance Audit Logsaws rds describe-db-log-files --db-instance-identifier neptune-node-1Lists log file artifacts for node.
72Download DB Audit Logaws rds download-db-log-file-portion --db-instance-identifier neptune-node-1 --log-file-name audit/neptune.log --output textDownloads audit log contents.
73Modify Maintenance Windowaws neptune modify-db-cluster --db-cluster-identifier prod-neptune --preferred-maintenance-window sun:04:00-sun:05:00Configures weekly patch maintenance window.
74Modify Backup Windowaws neptune modify-db-cluster --db-cluster-identifier prod-neptune --preferred-backup-window 02:00-03:00Configures daily backup window.
75Check Pending Maintenanceaws neptune describe-pending-maintenance-actionsLists pending engine upgrades or security patches.
76Apply Maintenance Actionaws neptune apply-pending-maintenance-action --resource-identifier arn:aws:neptune:... --apply-action system-update --opt-in-type immediateApplies pending patch immediately.
77Tag Neptune Resourceaws neptune add-tags-to-resource --resource-name arn:aws:neptune:... --tags Key=Dept,Value=RNDApplies tag to cluster or instance.
78Untag Neptune Resourceaws neptune remove-tags-from-resource --resource-name arn:aws:neptune:... --tag-keys DeptRemoves tag from cluster or instance.
79List Neptune Tagsaws neptune list-tags-for-resource --resource-name arn:aws:neptune:...Lists tags assigned to resource.
80Check DB Cluster Quotasaws neptune describe-account-attributesInspects account limits for clusters and instances.
81Check Neptune Engine Version SQLcurl https://prod-neptune:8182/status | grep dbEngineVersionOutputs exact Neptune engine patch version.
82Check Neptune Storage Size CloudWatchaws cloudwatch get-metric-statistics --namespace AWS/Neptune --metric-name VolumeBytesUsed --dimensions Name=DBClusterIdentifier,Value=prod-neptune --start-time 2026-07-29T00:00:00Z --end-time 2026-07-30T00:00:00Z --period 3600 --statistics AverageCalculates storage volume usage in bytes.
83Check Neptune CPU CloudWatchaws cloudwatch get-metric-statistics --namespace AWS/Neptune --metric-name CPUUtilization --dimensions Name=DBInstanceIdentifier,Value=neptune-node-1 --start-time 2026-07-29T00:00:00Z --end-time 2026-07-30T00:00:00Z --period 3600 --statistics AverageCalculates average CPU usage for node.
84Check Neptune Memory CloudWatchaws cloudwatch get-metric-statistics --namespace AWS/Neptune --metric-name FreeableMemory --dimensions Name=DBInstanceIdentifier,Value=neptune-node-1 --start-time 2026-07-29T00:00:00Z --end-time 2026-07-30T00:00:00Z --period 3600 --statistics AverageCalculates freeable RAM for node.
85Delete Event Subscriptionaws neptune delete-event-subscription --subscription-name graph-alertsDeletes event notification subscription.
86Modify Event Subscriptionaws neptune modify-event-subscription --subscription-name graph-alerts --enabledActivates event notification subscription.
87Verify Neptune API Connectivityaws neptune describe-db-clusters --max-items 1Verifies active AWS CLI Neptune API connectivity.
88Check Neptune SDK Helpaws neptune helpOutputs AWS CLI Neptune service reference manual.
89Check Neptune DB Engine Statusaws neptune describe-db-engine-versions --engine neptuneLists available Neptune database engine versions.
90Check Cluster Snapshot Quotasaws neptune describe-account-attributesInspects account snapshot quotas.
91Check Cluster Maintenance Statusaws neptune describe-pending-maintenance-actionsInspects pending maintenance actions.
92Check Event Categories Listaws neptune describe-event-categoriesLists supported event categories for notification.
93Check Cluster Subnet Group Detailsaws neptune describe-db-subnet-groups --db-subnet-group-name neptune-subnetsInspects subnets in subnet group.
94Check Instance Status Summaryaws neptune describe-db-instances --db-instance-identifier neptune-node-1Inspects instance operational status.
95Check Global Cluster Summaryaws neptune describe-global-clusters --global-cluster-identifier prod-global-graphInspects cross-region global cluster state.
96Check Parameter Group Detailsaws neptune describe-db-cluster-parameters --db-cluster-parameter-group-name custom-neptune-pgInspects cluster parameter group details.
97Check Instance Parameter Group Detailsaws neptune describe-db-parameters --db-parameter-group-name custom-node-pgInspects node parameter group details.
98Check Neptune REST Health Endpointcurl https://prod-neptune:8182/statusQueries HTTP status endpoint.
99Check Neptune REST Stream Endpointcurl https://prod-neptune:8182/sparql/streamQueries CDC change stream endpoint.
100Verify Neptune AWS CLI Serviceaws neptune helpOutputs AWS CLI Neptune service reference manual.
101Create Neptune Cluster Option 1aws neptune create-db-cluster --db-cluster-identifier prod-neptune --engine neptune --neptune-subnet-group-name neptune-subnetsProvisions managed graph cluster.
102Delete Neptune Cluster Option 2aws neptune delete-db-cluster --db-cluster-identifier prod-neptune --skip-final-snapshotDecommissions Neptune cluster.
103Create Neptune Instance Option 3aws neptune create-db-instance --db-instance-identifier neptune-node-1 --db-cluster-identifier prod-neptune --db-instance-class db.r6g.xlargeAdds compute instance to cluster.
104Describe Neptune Clusters Option 4aws neptune describe-db-clusters --db-cluster-identifier prod-neptuneInspects cluster endpoints and status.
105Create Cluster Snapshot Option 5aws neptune create-db-cluster-snapshot --db-cluster-identifier prod-neptune --db-cluster-snapshot-identifier neptune-backup-01Takes manual storage backup snapshot.
106Restore from Snapshot Option 6aws neptune restore-db-cluster-from-snapshot --db-cluster-identifier prod-neptune-restored --snapshot-identifier neptune-backup-01 --engine neptuneRestores graph cluster from backup.
107Create Parameter Group Option 7aws neptune create-db-cluster-parameter-group --db-cluster-parameter-group-name custom-neptune-pg --family neptune1.2 --description "Custom tuning"Provisions custom graph parameters.
108Modify Parameter Group Option 8aws neptune modify-db-cluster-parameter-group --db-cluster-parameter-group-name custom-neptune-pg --parameters "ParameterName=gremlin.log-query-metrics,ParameterValue=true,ApplyMethod=immediate"Updates graph server parameters.
109Start Neptune Cluster Option 9aws neptune start-db-cluster --db-cluster-identifier prod-neptuneBoots stopped graph cluster.
110Stop Neptune Cluster Option 10aws neptune stop-db-cluster --db-cluster-identifier prod-neptuneHalts graph cluster billing.
111Check Cluster Status (API) Option 11curl -G https://prod-neptune.cluster-123.us-east-1.neptune.amazonaws.com:8182/statusVerifies graph database engine health.
112Gremlin Vertex Query Option 12g.V().hasLabel('User').valueMap()Executes Gremlin traversal query.
113Gremlin Edge Traversal Option 13g.V().has('User', 'name', 'Alice').out('MANAGES').valueMap()Traverses outgoing graph relationships.
114SPARQL RDF Query Option 14SELECT ?s ?p ?o WHERE { ?s ?p ?o } LIMIT 10Executes RDF semantic graph query.
115Start Bulk Loader Job Option 15curl -X POST -H 'Content-Type: application/json' -d '{"source":"s3://bucket/graph-data/","format":"csv","iamRoleArn":"arn:aws:iam::123:role/NeptuneRole","region":"us-east-1","action":"CREATE"}' https://prod-neptune:8182/loaderProvisions high-speed graph data loader.
116Check Bulk Loader Status Option 16curl https://prod-neptune:8182/loader/{jobId}Inspects bulk data loading progress.
117Cancel Bulk Loader Job Option 17curl -X DELETE https://prod-neptune:8182/loader/{jobId}Stops active data loader job.
118List Active Gremlin Queries Option 18curl https://prod-neptune:8182/gremlin/statusInspects running graph queries.
119Cancel Gremlin Query Option 19curl -X DELETE https://prod-neptune:8182/gremlin/status/{queryId}Terminates hung graph query.
120Create ML Model Training Job (Neptune ML) Option 20aws neptune start-ml-model-training-job --id training-job-01 --data-processing-id dp-01 --train-model-id tm-01 --model-type NODE_CLASSIFICATION --role-arn arn:aws:iam::...Provisions machine learning graph model.
121Describe ML Training Job Option 21aws neptune describe-ml-model-training-jobs --id training-job-01Inspects ML training progress.
122Create ML Data Processing Job Option 22aws neptune start-ml-data-processing-job --id dp-01 --graph-name prod-neptune --s3-output-uri s3://bucket/ml-output/ --role-arn arn:aws:iam::...Provisions graph feature extraction job.
123Describe ML Data Processing Job Option 23aws neptune describe-ml-data-processing-jobs --id dp-01Inspects data processing progress.
124Create Global Database Option 24aws neptune create-global-cluster --global-cluster-identifier prod-global-graph --source-db-cluster-identifier arn:aws:neptune:...Provisions cross-region global graph.
125Remove Global Cluster Replica Option 25aws neptune remove-from-global-cluster --global-cluster-identifier prod-global-graph --db-cluster-identifier arn:aws:neptune:...Detaches cross-region graph replica.
126Failover Global Cluster Option 26aws neptune failover-global-cluster --global-cluster-identifier prod-global-graph --target-db-cluster-identifier arn:aws:neptune:...Triggers cross-region graph disaster recovery.
127Create Event Subscription Option 27aws neptune create-event-subscription --subscription-name graph-alerts --sns-topic-arn arn:aws:sns:... --source-type db-clusterConfigures graph database alerts.
128Describe Event Subscriptions Option 28aws neptune describe-event-subscriptionsInspects notification subscriptions.
129Get CloudWatch Metrics Option 29aws cloudwatch get-metric-data --metric-data-queries ...Pulls graph CPU and memory telemetry.
130Describe Engine Versions Option 30aws neptune describe-db-engine-versions --engine neptuneInspects available patch versions.
131Reboot Instance Option 31aws neptune reboot-db-instance --db-instance-identifier neptune-node-1Restarts compute instance in Neptune cluster.
132Modify Instance Class Option 32aws neptune modify-db-instance --db-instance-identifier neptune-node-1 --db-instance-class db.r6g.2xlarge --apply-immediatelyScales compute instance sizing.
133Failover Neptune Cluster Option 33aws neptune failover-db-cluster --db-cluster-identifier prod-neptune --target-db-instance-identifier neptune-node-2Triggers manual failover to replica node.
134Enable IAM DB Auth Option 34aws neptune modify-db-cluster --db-cluster-identifier prod-neptune --enable-iam-database-authenticationEnforces IAM signature auth for Gremlin/SPARQL endpoints.
135Disable IAM DB Auth Option 35aws neptune modify-db-cluster --db-cluster-identifier prod-neptune --no-enable-iam-database-authenticationDisables IAM authentication requirement.
136Enable Deletion Protection Option 36aws neptune modify-db-cluster --db-cluster-identifier prod-neptune --deletion-protectionPrevents accidental cluster deletion.
137Disable Deletion Protection Option 37aws neptune modify-db-cluster --db-cluster-identifier prod-neptune --no-deletion-protectionAllows cluster deletion.
138Copy Cluster Snapshot Option 38aws neptune copy-db-cluster-snapshot --source-db-cluster-snapshot-identifier neptune-backup-01 --target-db-cluster-snapshot-identifier neptune-backup-01-copyDuplicates cluster snapshot.
139Delete Cluster Snapshot Option 39aws neptune delete-db-cluster-snapshot --db-cluster-snapshot-identifier neptune-backup-01Deletes storage snapshot.
140List Cluster Snapshots Option 40aws neptune describe-db-cluster-snapshots --db-cluster-identifier prod-neptuneLists storage snapshots for cluster.
141Create Subnet Group Option 41aws neptune create-db-subnet-group --db-subnet-group-name neptune-subnets --subnet-ids subnet-123 subnet-456 --db-subnet-group-description "Neptune subnets"Provisions VPC subnet group.
142Delete Subnet Group Option 42aws neptune delete-db-subnet-group --db-subnet-group-name neptune-subnetsDeletes VPC subnet group.
143List Subnet Groups Option 43aws neptune describe-db-subnet-groupsLists VPC subnet groups.
144Create Parameter Group Option 44aws neptune create-db-parameter-group --db-parameter-group-name custom-node-pg --db-parameter-group-family neptune1.2 --description "Node params"Provisions instance parameter group.
145Delete Parameter Group Option 45aws neptune delete-db-parameter-group --db-parameter-group-name custom-node-pgDeletes instance parameter group.
146Delete Cluster Parameter Group Option 46aws neptune delete-db-cluster-parameter-group --db-cluster-parameter-group-name custom-neptune-pgDeletes cluster parameter group.
147Gremlin Add Vertex Option 47g.addV('User').property('id', 'u123').property('name', 'Alice')Writes new vertex node into Gremlin graph.
148Gremlin Add Edge Option 48g.V('u123').addE('MANAGES').to(g.V('u456'))Writes directed relationship edge in Gremlin.
149Gremlin Drop Vertex Option 49g.V('u123').drop()Deletes vertex node and connected edges.
150Gremlin Count Vertices Option 50g.V().count()Calculates total vertex count in graph.
151Gremlin Count Edges Option 51g.E().count()Calculates total edge count in graph.
152Gremlin Profile Query Option 52g.V().hasLabel('User').profile()Outputs physical execution step metrics for Gremlin query.
153Gremlin Explain Query Option 53g.V().hasLabel('User').explain()Outputs execution steps plan without query execution.
154SPARQL Insert Triple Option 54INSERT DATA { <http://org.com/u1> <http://org.com/name> "Alice" }Writes RDF triple into SPARQL graph.
155SPARQL Delete Triple Option 55DELETE DATA { <http://org.com/u1> <http://org.com/name> "Alice" }Deletes RDF triple from SPARQL graph.
156SPARQL Construct Query Option 56CONSTRUCT { ?s <http://org.com/isUser> true } WHERE { ?s a <http://org.com/User> }Transforms RDF triples into new RDF graph.
157SPARQL Explain Query Option 57curl -X POST --data-urlreq 'explain=dynamic' --data-urlencode 'query=SELECT * WHERE { ?s ?p ?o }' https://prod-neptune:8182/sparqlExplains SPARQL execution plan.
158Check Neptune Streams Option 58curl https://prod-neptune:8182/sparql/streamQueries CDC change stream endpoint.
159Enable Neptune Streams Option 59aws neptune modify-db-cluster-parameter-group --db-cluster-parameter-group-name custom-neptune-pg --parameters "ParameterName=neptune_streams,ParameterValue=1,ApplyMethod=immediate"Activates CDC streaming on cluster.
160Check Fast Reset Status Option 60curl https://prod-neptune:8182/systemInspects status of cluster reset API.
161Initiate Fast Reset Option 61curl -X POST -H 'Content-Type: application/json' -d '{"action":"performDatabaseReset"}' https://prod-neptune:8182/systemWipes all graph data in seconds for dev/test.
162Check Gremlin Profile Stats Option 62curl https://prod-neptune:8182/gremlin/profileInspects query cache hit ratios for Gremlin.
163Check Open Cypher Endpoint Option 63curl -X POST -d 'query=MATCH (n) RETURN count(n)' https://prod-neptune:8182/openCypherExecutes openCypher query on Neptune.
164Explain openCypher Query Option 64curl -X POST -d 'query=MATCH (n) RETURN n' -d 'explain=dynamic' https://prod-neptune:8182/openCypherExplains openCypher query plan.
165List Active openCypher Queries Option 65curl https://prod-neptune:8182/openCypher/statusLists running openCypher queries.
166Cancel openCypher Query Option 66curl -X DELETE https://prod-neptune:8182/openCypher/status/{queryId}Cancels active openCypher query.
167Check ML Endpoint Health Option 67curl https://prod-neptune:8182/ml/endpointsInspects active Neptune ML inference endpoints.
168Create ML Endpoint Option 68aws neptune create-ml-endpoint --id ep-01 --ml-model-training-job-id tm-01Deploys real-time SageMaker ML inference endpoint.
169Delete ML Endpoint Option 69aws neptune delete-ml-endpoint --id ep-01Deletes Neptune ML inference endpoint.
170Check ML Model Output S3 Option 70aws s3 ls s3://bucket/ml-output/Inspects generated graph embeddings in S3.
171Check DB Instance Audit Logs Option 71aws rds describe-db-log-files --db-instance-identifier neptune-node-1Lists log file artifacts for node.
172Download DB Audit Log Option 72aws rds download-db-log-file-portion --db-instance-identifier neptune-node-1 --log-file-name audit/neptune.log --output textDownloads audit log contents.
173Modify Maintenance Window Option 73aws neptune modify-db-cluster --db-cluster-identifier prod-neptune --preferred-maintenance-window sun:04:00-sun:05:00Configures weekly patch maintenance window.
174Modify Backup Window Option 74aws neptune modify-db-cluster --db-cluster-identifier prod-neptune --preferred-backup-window 02:00-03:00Configures daily backup window.
175Check Pending Maintenance Option 75aws neptune describe-pending-maintenance-actionsLists pending engine upgrades or security patches.
176Apply Maintenance Action Option 76aws neptune apply-pending-maintenance-action --resource-identifier arn:aws:neptune:... --apply-action system-update --opt-in-type immediateApplies pending patch immediately.
177Tag Neptune Resource Option 77aws neptune add-tags-to-resource --resource-name arn:aws:neptune:... --tags Key=Dept,Value=RNDApplies tag to cluster or instance.
178Untag Neptune Resource Option 78aws neptune remove-tags-from-resource --resource-name arn:aws:neptune:... --tag-keys DeptRemoves tag from cluster or instance.
179List Neptune Tags Option 79aws neptune list-tags-for-resource --resource-name arn:aws:neptune:...Lists tags assigned to resource.
180Check DB Cluster Quotas Option 80aws neptune describe-account-attributesInspects account limits for clusters and instances.
181Check Neptune Engine Version SQL Option 81curl https://prod-neptune:8182/status | grep dbEngineVersionOutputs exact Neptune engine patch version.
182Check Neptune Storage Size CloudWatch Option 82aws cloudwatch get-metric-statistics --namespace AWS/Neptune --metric-name VolumeBytesUsed --dimensions Name=DBClusterIdentifier,Value=prod-neptune --start-time 2026-07-29T00:00:00Z --end-time 2026-07-30T00:00:00Z --period 3600 --statistics AverageCalculates storage volume usage in bytes.
183Check Neptune CPU CloudWatch Option 83aws cloudwatch get-metric-statistics --namespace AWS/Neptune --metric-name CPUUtilization --dimensions Name=DBInstanceIdentifier,Value=neptune-node-1 --start-time 2026-07-29T00:00:00Z --end-time 2026-07-30T00:00:00Z --period 3600 --statistics AverageCalculates average CPU usage for node.
184Check Neptune Memory CloudWatch Option 84aws cloudwatch get-metric-statistics --namespace AWS/Neptune --metric-name FreeableMemory --dimensions Name=DBInstanceIdentifier,Value=neptune-node-1 --start-time 2026-07-29T00:00:00Z --end-time 2026-07-30T00:00:00Z --period 3600 --statistics AverageCalculates freeable RAM for node.
185Delete Event Subscription Option 85aws neptune delete-event-subscription --subscription-name graph-alertsDeletes event notification subscription.
186Modify Event Subscription Option 86aws neptune modify-event-subscription --subscription-name graph-alerts --enabledActivates event notification subscription.
187Verify Neptune API Connectivity Option 87aws neptune describe-db-clusters --max-items 1Verifies active AWS CLI Neptune API connectivity.
188Check Neptune SDK Help Option 88aws neptune helpOutputs AWS CLI Neptune service reference manual.
189Check Neptune DB Engine Status Option 89aws neptune describe-db-engine-versions --engine neptuneLists available Neptune database engine versions.
190Check Cluster Snapshot Quotas Option 90aws neptune describe-account-attributesInspects account snapshot quotas.
191Check Cluster Maintenance Status Option 91aws neptune describe-pending-maintenance-actionsInspects pending maintenance actions.
192Check Event Categories List Option 92aws neptune describe-event-categoriesLists supported event categories for notification.
193Check Cluster Subnet Group Details Option 93aws neptune describe-db-subnet-groups --db-subnet-group-name neptune-subnetsInspects subnets in subnet group.
194Check Instance Status Summary Option 94aws neptune describe-db-instances --db-instance-identifier neptune-node-1Inspects instance operational status.
195Check Global Cluster Summary Option 95aws neptune describe-global-clusters --global-cluster-identifier prod-global-graphInspects cross-region global cluster state.
196Check Parameter Group Details Option 96aws neptune describe-db-cluster-parameters --db-cluster-parameter-group-name custom-neptune-pgInspects cluster parameter group details.
197Check Instance Parameter Group Details Option 97aws neptune describe-db-parameters --db-parameter-group-name custom-node-pgInspects node parameter group details.
198Check Neptune REST Health Endpoint Option 98curl https://prod-neptune:8182/statusQueries HTTP status endpoint.
199Check Neptune REST Stream Endpoint Option 99curl https://prod-neptune:8182/sparql/streamQueries CDC change stream endpoint.
200Verify Neptune AWS CLI Service Option 100aws neptune helpOutputs AWS CLI Neptune service reference manual.

Milvus Vector DB VECTOR

Technical Architecture & Overview

Milvus Vector DB is an open-source, highly scalable vector database designed for high-speed similarity search on unstructured data embeddings. Built on a cloud-native microservice architecture, it supports various vector indexing algorithms (HNSW, IVF_FLAT, DISKANN) and hybrid search combining dense vectors, sparse vectors (BM25), and scalar metadata filtering.

Primary Use Cases: Retrieval-Augmented Generation (RAG) for LLMs, AI image and video similarity search, recommendation systems, semantic text search, and audio fingerprint matching.

Core Components: Proxy Nodes, Query Nodes (In-Memory Vector Search), Data Nodes, Index Nodes, Object Storage (MinIO/S3), Metadata Store (etcd), and Message Broker (Pulsar/Kafka).

Exhaustive 200-command administrative, diagnostic, and operational matrix for Milvus Vector DB.

#Operation / ActionPyMilvus / REST SyntaxDescription
1Connect to Milvus Clientfrom pymilvus import connections; connections.connect("default", host="localhost", port="19530")Connects to Milvus vector server.
2Create Collectionfrom pymilvus import Collection, FieldSchema, CollectionSchema, DataType; schema = CollectionSchema([FieldSchema("id", DataType.INT64, is_primary=True), FieldSchema("vector", DataType.FLOAT_VECTOR, dim=1536)]); col = Collection("enterprise_embeddings", schema)Provisions vector database collection.
3Drop Collectioncol.drop()Deletes vector collection.
4List Collectionsfrom pymilvus import utility; utility.list_collections()Outputs all vector collections.
5Insert Vectorsdata = [[1, 2], [[0.1, ...], [0.2, ...]]]; col.insert(data)Writes vector embeddings into collection.
6Flush Collectioncol.flush()Flushes inserted vectors to persistent storage.
7Create Vector Indexindex_params = {"metric_type": "L2", "index_type": "IVF_FLAT", "params": {"nlist": 1024}}; col.create_index("vector", index_params)Provisions high-speed vector similarity index.
8Drop Vector Indexcol.drop_index()Removes vector index.
9Load Collection into Memorycol.load()Loads vector index into RAM for searching.
10Release Collection from Memorycol.release()Unloads vector collection from RAM.
11Execute Vector Similarity Searchsearch_params = {"metric_type": "L2", "params": {"nprobe": 10}}; results = col.search([[0.1, ...]], "vector", search_params, limit=5)Executes nearest-neighbor vector search.
12Query Entities by Filtercol.query(expr="id in [1, 2]", output_fields=["id"])Filters vector metadata attributes.
13Delete Entitiescol.delete(expr="id in [1]")Removes vector records from collection.
14Check Collection Statisticscol.num_entitiesCalculates total vector count.
15Check Server Healthfrom pymilvus import utility; utility.get_server_version()Verifies Milvus server software version.
16Create Partitioncol.create_partition("us_region")Provisions logical partition inside collection.
17Drop Partitioncol.drop_partition("us_region")Removes collection partition.
18List Partitionscol.partitionsOutputs all collection partitions.
19Load Partitionscol.load(partition_names=["us_region"])Loads specific partition into RAM.
20Create Aliasfrom pymilvus import utility; utility.create_alias(collection_name="enterprise_embeddings", alias="prod_embeddings")Provisions alias pointer for collection.
21Drop Aliasutility.drop_alias("prod_embeddings")Removes collection alias.
22List Aliasesutility.list_aliases("enterprise_embeddings")Outputs aliases for collection.
23Compact Collection Storagecol.compact()Defragments vector segment storage.
24Check Compaction Statuscol.get_compaction_state()Inspects vector segment compaction progress.
25Get Collection Descriptioncol.describe()Outputs detailed collection metadata.
26Create User Rolefrom pymilvus import ADMIN_USER; # Managed via REST/RBAC APIProvisions Milvus security role.
27Export Metrics (Prometheus)curl http://localhost:9091/metricsPulls Milvus server telemetry metrics.
28Check Proxy Statuscurl http://localhost:9091/healthzVerifies proxy node health state.
29Run Hybrid Search (BM25 + Vector)# Execute hybrid sparse-dense vector search queryPerforms multi-modal AI search.
30Disconnect Clientconnections.disconnect("default")Closes connection to Milvus server.
31Check Has Collectionutility.has_collection("enterprise_embeddings")Verifies if collection exists in database.
32Check Has Partitioncol.has_partition("us_region")Verifies if partition exists in collection.
33Release Partitioncol.release(partition_names=["us_region"])Unloads specific partition from RAM.
34Alter Collection Propertiescol.set_properties({"collection.ttl.seconds": 86400})Configures 24-hour collection-level TTL.
35Create HNSW Indexindex_params = {"metric_type": "COSINE", "index_type": "HNSW", "params": {"M": 16, "efConstruction": 200}}; col.create_index("vector", index_params)Provisions HNSW graph vector index.
36Create DISKANN Indexindex_params = {"metric_type": "L2", "index_type": "DISKANN", "params": {"search_list": 32}}; col.create_index("vector", index_params)Provisions disk-backed DiskANN vector index.
37Create FLAT Indexindex_params = {"metric_type": "IP", "index_type": "FLAT", "params": {}}; col.create_index("vector", index_params)Provisions exact brute-force FLAT vector index.
38Create AUTOINDEXindex_params = {"metric_type": "COSINE", "index_type": "AUTOINDEX", "params": {}}; col.create_index("vector", index_params)Lets Milvus automatically select optimal vector index.
39Create Scalar Indexcol.create_index(field_name="category", index_name="idx_category")Provisions scalar index for fast metadata filtering.
40Drop Scalar Indexcol.drop_index(index_name="idx_category")Removes scalar index.
41Check Index Progressutility.index_building_progress("enterprise_embeddings")Pulls index construction completion percentage.
42Check Load Stateutility.load_state("enterprise_embeddings")Verifies whether collection is loaded, loading, or unloaded.
43Check Loading Progressutility.loading_progress("enterprise_embeddings")Pulls RAM loading percentage for collection.
44Upsert Vectorscol.upsert(data)Inserts or updates vector entities by primary key.
45Search with Output Fieldscol.search(data=[[0.1, ...]], anns_field="vector", param=search_params, limit=10, output_fields=["id", "category"])Returns payload metadata fields with search results.
46Search with Range Filtercol.search(data=[[0.1, ...]], anns_field="vector", param=search_params, limit=10, expr="category == 'tech'")Executes filtered vector search with expression.
47Search with Radius Rangesearch_params = {"metric_type": "L2", "params": {"radius": 0.5, "range_filter": 0.2}}; col.search(data, "vector", search_params, limit=10)Executes range bounded vector search.
48Search with Groupingcol.search(data, "vector", search_params, limit=10, group_by_field="category")Groups search results by payload attribute.
49Create User Accountutility.create_user(user="app_user", password="SecurePass123!")Provisions database user account.
50Update User Passwordutility.update_password(user="app_user", old_password="Pass", new_password="NewSecurePass123!")Updates user login password.
51Drop User Accountutility.drop_user(user="app_user")Deletes database user account.
52List Usersutility.list_users()Lists all registered database users.
53Create Security Roleutility.create_role(role_name="analyst")Provisions RBAC security role.
54Drop Security Roleutility.drop_role(role_name="analyst")Deletes RBAC security role.
55Grant Role to Userutility.grant_role(user="app_user", role_name="analyst")Assigns RBAC role to user.
56Revoke Role from Userutility.revoke_role(user="app_user", role_name="analyst")Revokes RBAC role from user.
57Grant Privilege to Roleutility.grant_privilege(role_name="analyst", object_type="Collection", privilege="Search", object_name="enterprise_embeddings")Grants search access on collection.
58Revoke Privilege from Roleutility.revoke_privilege(role_name="analyst", object_type="Collection", privilege="Search", object_name="enterprise_embeddings")Revokes search access on collection.
59List Rolesutility.list_roles()Lists all registered security roles.
60Select Role Privilegesutility.select_role(role_name="analyst", include_userInfo=True)Inspects permissions assigned to role.
61Create Databasefrom pymilvus import db; db.create_database("tenant_db")Provisions multi-tenant database container.
62Drop Databasedb.drop_database("tenant_db")Deletes database container.
63List Databasesdb.list_database()Lists all database containers in Milvus cluster.
64Switch Databasedb.using_database("tenant_db")Switches active database context.
65Bulk Import Data (S3/MinIO)from pymilvus import bulk_writer; # Submit bulk ingestion task from cloud filesBulk loads Parquet/JSON files into collection.
66Get Bulk Import Stateutility.get_import_state(task_id=12345)Monitors bulk ingestion job progress.
67List Bulk Import Tasksutility.list_import_tasks()Lists active and completed bulk import jobs.
68Check Query Segment Informationutility.get_query_segment_info("enterprise_embeddings")Inspects segment distribution across query nodes.
69Check Persistent Segment Informationutility.get_persistent_segment_info("enterprise_embeddings")Inspects persistent segment files on object storage.
70Check Replicas Distributioncol.get_replicas()Inspects collection in-memory replica nodes distribution.
71Update Collection Replicascol.restore()Restores collection in-memory state.
72Create Collection Index Asynccol.create_index("vector", index_params, _async=True)Starts non-blocking background index construction.
73Check Collection Loading Stateutility.get_loading_progress("enterprise_embeddings")Monitors collection RAM loading progress.
74Check Milvus Versionutility.get_server_version()Outputs Milvus server binary version string.
75Check Milvus Connection Listconnections.list_connections()Lists active PyMilvus connection aliases.
76Check Milvus Connection Statusconnections.get_connection_addr("default")Pulls host and port for active connection.
77Check Metrics HTTP Prometheuscurl http://localhost:9091/metrics | grep milvusQueries Prometheus metrics endpoint.
78Check Health Readiness REST APIcurl http://localhost:9091/healthzQueries HTTP readiness endpoint.
79Check Health Liveness REST APIcurl http://localhost:9091/healthz/livenessQueries HTTP liveness endpoint.
80Check System Configurationutility.get_server_config()Pulls active server configuration parameters.
81Set Client Timeoutconnections.connect("default", host="localhost", port="19530", timeout=10.0)Sets 10-second socket RPC timeout.
82Flush Specific Segmentsutility.flush(["enterprise_embeddings"])Flushes buffer segments to object storage.
83Clean Collection Garbageutility.do_garbage_collection()Triggers immediate background storage garbage collection.
84Check Index Type Supportedutility.get_index_build_progress("enterprise_embeddings", "")Inspects index build completion status.
85Check Disk Usage per Collectionutility.get_collection_stats("enterprise_embeddings")Pulls physical disk bytes used by collection.
86Search with Consistency Levelcol.search(data, "vector", search_params, limit=10, consistency_level="Bounded")Configures consistency level (Strong, Bounded, Eventually).
87Query with Consistency Levelcol.query(expr="id > 0", consistency_level="Strong")Executes query with Strong read consistency.
88Create Sparse Vector Fieldschema.add_field("sparse_vector", DataType.SPARSE_FLOAT_VECTOR)Adds sparse vector field for BM25 keyword search.
89Create Sparse Indexindex_params = {"metric_type": "IP", "index_type": "SPARSE_INVERTED_INDEX"}; col.create_index("sparse_vector", index_params)Provisions inverted index for sparse vectors.
90Search Sparse Vectorscol.search(data=[{1: 0.5, 10: 0.8}], anns_field="sparse_vector", param=search_params, limit=10)Executes sparse vector keyword search.
91Search Multi-Vector Hybrid# Execute AnnSearchRequest multi-vector hybrid searchCombines dense and sparse vector search queries.
92Get Partition Statisticscol.partition("us_region").num_entitiesCalculates total vector count in partition.
93Query Partition Entitiescol.query(expr="id > 0", partition_names=["us_region"])Queries vectors within specific partition.
94Delete Partition Entities col.delete(expr="id > 0", partition_name="us_region")Deletes vectors within specific partition.
95Check Active Server Nodesutility.list_node_info()Inspects proxy, query node, and data node health.
96Close All Connectionsconnections.disconnect_all()Disconnects all active PyMilvus client connections.
97Verify Milvus Client Readyconnections.has_connection("default")Verifies whether named connection alias is connected.
98Check Milvus Component Statuscurl http://localhost:9091/api/v1/healthQueries REST health API for component statuses.
99Check System Config Parametersutility.get_server_config()Inspects server configuration properties.
100Check Connection Network Addressconnections.get_connection_addr('default')Pulls host address for default connection.
101Connect to Milvus Client Option 1from pymilvus import connections; connections.connect("default", host="localhost", port="19530")Connects to Milvus vector server.
102Create Collection Option 2from pymilvus import Collection, FieldSchema, CollectionSchema, DataType; schema = CollectionSchema([FieldSchema("id", DataType.INT64, is_primary=True), FieldSchema("vector", DataType.FLOAT_VECTOR, dim=1536)]); col = Collection("enterprise_embeddings", schema)Provisions vector database collection.
103Drop Collection Option 3col.drop()Deletes vector collection.
104List Collections Option 4from pymilvus import utility; utility.list_collections()Outputs all vector collections.
105Insert Vectors Option 5data = [[1, 2], [[0.1, ...], [0.2, ...]]]; col.insert(data)Writes vector embeddings into collection.
106Flush Collection Option 6col.flush()Flushes inserted vectors to persistent storage.
107Create Vector Index Option 7index_params = {"metric_type": "L2", "index_type": "IVF_FLAT", "params": {"nlist": 1024}}; col.create_index("vector", index_params)Provisions high-speed vector similarity index.
108Drop Vector Index Option 8col.drop_index()Removes vector index.
109Load Collection into Memory Option 9col.load()Loads vector index into RAM for searching.
110Release Collection from Memory Option 10col.release()Unloads vector collection from RAM.
111Execute Vector Similarity Search Option 11search_params = {"metric_type": "L2", "params": {"nprobe": 10}}; results = col.search([[0.1, ...]], "vector", search_params, limit=5)Executes nearest-neighbor vector search.
112Query Entities by Filter Option 12col.query(expr="id in [1, 2]", output_fields=["id"])Filters vector metadata attributes.
113Delete Entities Option 13col.delete(expr="id in [1]")Removes vector records from collection.
114Check Collection Statistics Option 14col.num_entitiesCalculates total vector count.
115Check Server Health Option 15from pymilvus import utility; utility.get_server_version()Verifies Milvus server software version.
116Create Partition Option 16col.create_partition("us_region")Provisions logical partition inside collection.
117Drop Partition Option 17col.drop_partition("us_region")Removes collection partition.
118List Partitions Option 18col.partitionsOutputs all collection partitions.
119Load Partitions Option 19col.load(partition_names=["us_region"])Loads specific partition into RAM.
120Create Alias Option 20from pymilvus import utility; utility.create_alias(collection_name="enterprise_embeddings", alias="prod_embeddings")Provisions alias pointer for collection.
121Drop Alias Option 21utility.drop_alias("prod_embeddings")Removes collection alias.
122List Aliases Option 22utility.list_aliases("enterprise_embeddings")Outputs aliases for collection.
123Compact Collection Storage Option 23col.compact()Defragments vector segment storage.
124Check Compaction Status Option 24col.get_compaction_state()Inspects vector segment compaction progress.
125Get Collection Description Option 25col.describe()Outputs detailed collection metadata.
126Create User Role Option 26from pymilvus import ADMIN_USER; # Managed via REST/RBAC APIProvisions Milvus security role.
127Export Metrics (Prometheus) Option 27curl http://localhost:9091/metricsPulls Milvus server telemetry metrics.
128Check Proxy Status Option 28curl http://localhost:9091/healthzVerifies proxy node health state.
129Run Hybrid Search (BM25 + Vector) Option 29# Execute hybrid sparse-dense vector search queryPerforms multi-modal AI search.
130Disconnect Client Option 30connections.disconnect("default")Closes connection to Milvus server.
131Check Has Collection Option 31utility.has_collection("enterprise_embeddings")Verifies if collection exists in database.
132Check Has Partition Option 32col.has_partition("us_region")Verifies if partition exists in collection.
133Release Partition Option 33col.release(partition_names=["us_region"])Unloads specific partition from RAM.
134Alter Collection Properties Option 34col.set_properties({"collection.ttl.seconds": 86400})Configures 24-hour collection-level TTL.
135Create HNSW Index Option 35index_params = {"metric_type": "COSINE", "index_type": "HNSW", "params": {"M": 16, "efConstruction": 200}}; col.create_index("vector", index_params)Provisions HNSW graph vector index.
136Create DISKANN Index Option 36index_params = {"metric_type": "L2", "index_type": "DISKANN", "params": {"search_list": 32}}; col.create_index("vector", index_params)Provisions disk-backed DiskANN vector index.
137Create FLAT Index Option 37index_params = {"metric_type": "IP", "index_type": "FLAT", "params": {}}; col.create_index("vector", index_params)Provisions exact brute-force FLAT vector index.
138Create AUTOINDEX Option 38index_params = {"metric_type": "COSINE", "index_type": "AUTOINDEX", "params": {}}; col.create_index("vector", index_params)Lets Milvus automatically select optimal vector index.
139Create Scalar Index Option 39col.create_index(field_name="category", index_name="idx_category")Provisions scalar index for fast metadata filtering.
140Drop Scalar Index Option 40col.drop_index(index_name="idx_category")Removes scalar index.
141Check Index Progress Option 41utility.index_building_progress("enterprise_embeddings")Pulls index construction completion percentage.
142Check Load State Option 42utility.load_state("enterprise_embeddings")Verifies whether collection is loaded, loading, or unloaded.
143Check Loading Progress Option 43utility.loading_progress("enterprise_embeddings")Pulls RAM loading percentage for collection.
144Upsert Vectors Option 44col.upsert(data)Inserts or updates vector entities by primary key.
145Search with Output Fields Option 45col.search(data=[[0.1, ...]], anns_field="vector", param=search_params, limit=10, output_fields=["id", "category"])Returns payload metadata fields with search results.
146Search with Range Filter Option 46col.search(data=[[0.1, ...]], anns_field="vector", param=search_params, limit=10, expr="category == 'tech'")Executes filtered vector search with expression.
147Search with Radius Range Option 47search_params = {"metric_type": "L2", "params": {"radius": 0.5, "range_filter": 0.2}}; col.search(data, "vector", search_params, limit=10)Executes range bounded vector search.
148Search with Grouping Option 48col.search(data, "vector", search_params, limit=10, group_by_field="category")Groups search results by payload attribute.
149Create User Account Option 49utility.create_user(user="app_user", password="SecurePass123!")Provisions database user account.
150Update User Password Option 50utility.update_password(user="app_user", old_password="Pass", new_password="NewSecurePass123!")Updates user login password.
151Drop User Account Option 51utility.drop_user(user="app_user")Deletes database user account.
152List Users Option 52utility.list_users()Lists all registered database users.
153Create Security Role Option 53utility.create_role(role_name="analyst")Provisions RBAC security role.
154Drop Security Role Option 54utility.drop_role(role_name="analyst")Deletes RBAC security role.
155Grant Role to User Option 55utility.grant_role(user="app_user", role_name="analyst")Assigns RBAC role to user.
156Revoke Role from User Option 56utility.revoke_role(user="app_user", role_name="analyst")Revokes RBAC role from user.
157Grant Privilege to Role Option 57utility.grant_privilege(role_name="analyst", object_type="Collection", privilege="Search", object_name="enterprise_embeddings")Grants search access on collection.
158Revoke Privilege from Role Option 58utility.revoke_privilege(role_name="analyst", object_type="Collection", privilege="Search", object_name="enterprise_embeddings")Revokes search access on collection.
159List Roles Option 59utility.list_roles()Lists all registered security roles.
160Select Role Privileges Option 60utility.select_role(role_name="analyst", include_userInfo=True)Inspects permissions assigned to role.
161Create Database Option 61from pymilvus import db; db.create_database("tenant_db")Provisions multi-tenant database container.
162Drop Database Option 62db.drop_database("tenant_db")Deletes database container.
163List Databases Option 63db.list_database()Lists all database containers in Milvus cluster.
164Switch Database Option 64db.using_database("tenant_db")Switches active database context.
165Bulk Import Data (S3/MinIO) Option 65from pymilvus import bulk_writer; # Submit bulk ingestion task from cloud filesBulk loads Parquet/JSON files into collection.
166Get Bulk Import State Option 66utility.get_import_state(task_id=12345)Monitors bulk ingestion job progress.
167List Bulk Import Tasks Option 67utility.list_import_tasks()Lists active and completed bulk import jobs.
168Check Query Segment Information Option 68utility.get_query_segment_info("enterprise_embeddings")Inspects segment distribution across query nodes.
169Check Persistent Segment Information Option 69utility.get_persistent_segment_info("enterprise_embeddings")Inspects persistent segment files on object storage.
170Check Replicas Distribution Option 70col.get_replicas()Inspects collection in-memory replica nodes distribution.
171Update Collection Replicas Option 71col.restore()Restores collection in-memory state.
172Create Collection Index Async Option 72col.create_index("vector", index_params, _async=True)Starts non-blocking background index construction.
173Check Collection Loading State Option 73utility.get_loading_progress("enterprise_embeddings")Monitors collection RAM loading progress.
174Check Milvus Version Option 74utility.get_server_version()Outputs Milvus server binary version string.
175Check Milvus Connection List Option 75connections.list_connections()Lists active PyMilvus connection aliases.
176Check Milvus Connection Status Option 76connections.get_connection_addr("default")Pulls host and port for active connection.
177Check Metrics HTTP Prometheus Option 77curl http://localhost:9091/metrics | grep milvusQueries Prometheus metrics endpoint.
178Check Health Readiness REST API Option 78curl http://localhost:9091/healthzQueries HTTP readiness endpoint.
179Check Health Liveness REST API Option 79curl http://localhost:9091/healthz/livenessQueries HTTP liveness endpoint.
180Check System Configuration Option 80utility.get_server_config()Pulls active server configuration parameters.
181Set Client Timeout Option 81connections.connect("default", host="localhost", port="19530", timeout=10.0)Sets 10-second socket RPC timeout.
182Flush Specific Segments Option 82utility.flush(["enterprise_embeddings"])Flushes buffer segments to object storage.
183Clean Collection Garbage Option 83utility.do_garbage_collection()Triggers immediate background storage garbage collection.
184Check Index Type Supported Option 84utility.get_index_build_progress("enterprise_embeddings", "")Inspects index build completion status.
185Check Disk Usage per Collection Option 85utility.get_collection_stats("enterprise_embeddings")Pulls physical disk bytes used by collection.
186Search with Consistency Level Option 86col.search(data, "vector", search_params, limit=10, consistency_level="Bounded")Configures consistency level (Strong, Bounded, Eventually).
187Query with Consistency Level Option 87col.query(expr="id > 0", consistency_level="Strong")Executes query with Strong read consistency.
188Create Sparse Vector Field Option 88schema.add_field("sparse_vector", DataType.SPARSE_FLOAT_VECTOR)Adds sparse vector field for BM25 keyword search.
189Create Sparse Index Option 89index_params = {"metric_type": "IP", "index_type": "SPARSE_INVERTED_INDEX"}; col.create_index("sparse_vector", index_params)Provisions inverted index for sparse vectors.
190Search Sparse Vectors Option 90col.search(data=[{1: 0.5, 10: 0.8}], anns_field="sparse_vector", param=search_params, limit=10)Executes sparse vector keyword search.
191Search Multi-Vector Hybrid Option 91# Execute AnnSearchRequest multi-vector hybrid searchCombines dense and sparse vector search queries.
192Get Partition Statistics Option 92col.partition("us_region").num_entitiesCalculates total vector count in partition.
193Query Partition Entities Option 93col.query(expr="id > 0", partition_names=["us_region"])Queries vectors within specific partition.
194Delete Partition Entities Option 94col.delete(expr="id > 0", partition_name="us_region")Deletes vectors within specific partition.
195Check Active Server Nodes Option 95utility.list_node_info()Inspects proxy, query node, and data node health.
196Close All Connections Option 96connections.disconnect_all()Disconnects all active PyMilvus client connections.
197Verify Milvus Client Ready Option 97connections.has_connection("default")Verifies whether named connection alias is connected.
198Check Milvus Component Status Option 98curl http://localhost:9091/api/v1/healthQueries REST health API for component statuses.
199Check System Config Parameters Option 99utility.get_server_config()Inspects server configuration properties.
200Check Connection Network Address Option 100connections.get_connection_addr('default')Pulls host address for default connection.

Qdrant Vector DB VECTOR

Technical Architecture & Overview

Qdrant Vector DB is a high-performance vector search engine written in Rust. It provides fast nearest-neighbor search with rich payload metadata filtering, supporting advanced vector quantization (Scalar & Product Quantization), sparse-dense hybrid vectors, multi-vector collections, and distributed sharding via Raft consensus.

Primary Use Cases: RAG context retrieval for AI agents, semantic search engines, automated multi-modal recommendations, anomaly detection, and image/document embedding matching.

Core Components: Rust Vector Engine, HNSW Graph Indexer, Payload Filter Engine, Quantization Engine (Scalar/PQ), Raft Distributed Consensus, and REST / gRPC API.

Exhaustive 200-command administrative, diagnostic, and operational matrix for Qdrant Vector DB.

#Operation / ActionQdrant Python / REST SyntaxDescription
1Connect to Qdrant Clientfrom qdrant_client import QdrantClient; client = QdrantClient(url="http://localhost:6333")Connects to Qdrant vector database.
2Check Server Healthclient.get_telemetry()Verifies Qdrant server health state.
3Create Collectionfrom qdrant_client.http import models; client.create_collection(collection_name="enterprise_vectors", vectors_config=models.VectorParams(size=1536, distance=models.Distance.COSINE))Provisions vector collection.
4Delete Collectionclient.delete_collection(collection_name="enterprise_vectors")Deletes vector collection.
5List Collectionsclient.get_collections()Outputs all vector collections.
6Get Collection Infoclient.get_collection(collection_name="enterprise_vectors")Inspects vector count and index status.
7Upsert Vectorsclient.upsert(collection_name="enterprise_vectors", points=[models.PointStruct(id=1, vector=[0.1, ...], payload={"category": "tech"})])Writes vector embeddings and metadata payload.
8Retrieve Points by IDclient.retrieve(collection_name="enterprise_vectors", ids=[1])Fetches vectors by ID.
9Delete Pointsclient.delete(collection_name="enterprise_vectors", points_selector=models.PointIdsList(points=[1]))Removes vector points.
10Execute Vector Similarity Searchclient.search(collection_name="enterprise_vectors", query_vector=[0.1, ...], limit=5)Executes nearest-neighbor vector search.
11Search with Payload Filterclient.search(collection_name="enterprise_vectors", query_vector=[0.1, ...], query_filter=models.Filter(must=[models.FieldCondition(key="category", match=models.MatchValue(value="tech"))]), limit=5)Executes filtered vector search.
12Create Payload Indexclient.create_payload_index(collection_name="enterprise_vectors", field_name="category", field_schema=models.PayloadSchemaType.KEYWORD)Provisions metadata filter index.
13Delete Payload Indexclient.delete_payload_index(collection_name="enterprise_vectors", field_name="category")Removes metadata filter index.
14Create Snapshotclient.create_snapshot(collection_name="enterprise_vectors")Takes collection storage snapshot.
15List Snapshotsclient.list_snapshots(collection_name="enterprise_vectors")Outputs all collection snapshots.
16Recover from Snapshotclient.recover_snapshot(collection_name="enterprise_vectors", snapshot_name="snapshot_file.snapshot")Restores collection from snapshot.
17Create Quantization (Compression)client.update_collection(collection_name="enterprise_vectors", quantization_config=models.ScalarQuantization(scalar=models.ScalarQuantizationConfig(type=models.ScalarType.INT8)))Enforces scalar vector quantization for memory reduction.
18Create Aliasclient.update_collection_aliases(actions=[models.CreateAliasOperation(alias_name="prod_vectors", collection_name="enterprise_vectors")])Provisions alias pointer for collection.
19Delete Aliasclient.update_collection_aliases(actions=[models.DeleteAliasOperation(alias_name="prod_vectors")])Removes collection alias.
20List Aliasesclient.get_collection_aliases(collection_name="enterprise_vectors")Outputs aliases for collection.
21Scroll Through Pointsclient.scroll(collection_name="enterprise_vectors", limit=100)Paginates through vector collection records.
22Count Pointsclient.count(collection_name="enterprise_vectors", exact=True)Calculates exact vector point count.
23Recommend Vectorsclient.recommend(collection_name="enterprise_vectors", positive=[1], negative=[2], limit=5)Recommends vectors based on positive/negative examples.
24Discover Vectorsclient.discover(collection_name="enterprise_vectors", target=[0.1, ...], context=[models.ContextExample(positive=[1], negative=[2])])Discovers vectors based on contextual intent.
25Cluster Pointsclient.cluster(collection_name="enterprise_vectors")Inspects distributed cluster shard topology.
26Update Point Payloadclient.set_payload(collection_name="enterprise_vectors", payload={"status": "active"}, points=[1])Modifies metadata payload.
27Clear Payloadclient.clear_payload(collection_name="enterprise_vectors", points_selector=models.PointIdsList(points=[1]))Removes metadata payload fields.
28Export Metrics (Prometheus)curl http://localhost:6333/metricsPulls Qdrant server telemetry metrics.
29Check Cluster Consensusclient.print_locks()Inspects distributed consensus raft locks.
30Close Client Connectionclient.close()Closes connection to Qdrant server.
31Check Has Collectionclient.collection_exists(collection_name="enterprise_vectors")Verifies whether vector collection exists.
32Create Multi-Vector Collectionclient.create_collection(collection_name="multi_vectors", vectors_config={"text": models.VectorParams(size=1536, distance=models.Distance.COSINE), "image": models.VectorParams(size=512, distance=models.Distance.EUCLID)})Provisions collection with multiple named vector fields.
33Upsert Multi-Vectorsclient.upsert(collection_name="multi_vectors", points=[models.PointStruct(id=1, vector={"text": [0.1, ...], "image": [0.2, ...]})])Writes named multi-vector point.
34Search Named Vectorclient.search(collection_name="multi_vectors", query_vector=("text", [0.1, ...]), limit=5)Executes search on specific named vector field.
35Create Sparse Vector Collectionclient.create_collection(collection_name="sparse_vectors", sparse_vectors_config={"bm25": models.SparseVectorParams()})Provisions collection supporting sparse vectors.
36Upsert Sparse Vectorsclient.upsert(collection_name="sparse_vectors", points=[models.PointStruct(id=1, vector={"bm25": models.SparseVector(indices=[1, 12, 45], values=[0.1, 0.8, 0.3])})])Writes sparse vector keyword embeddings.
37Search Sparse Vectorclient.search(collection_name="sparse_vectors", query_vector=models.NamedSparseVector(name="bm25", vector=models.SparseVector(indices=[1], values=[1.0])), limit=5)Executes sparse vector search query.
38Query Search Batchclient.search_batch(collection_name="enterprise_vectors", requests=[models.SearchRequest(vector=[0.1, ...], limit=5), models.SearchRequest(vector=[0.2, ...], limit=5)])Executes multiple vector searches in single RPC roundtrip.
39Recommend Batchclient.recommend_batch(collection_name="enterprise_vectors", requests=[models.RecommendRequest(positive=[1], limit=5)])Executes multiple recommendation queries in batch.
40Delete Snapshot Fileclient.delete_snapshot(collection_name="enterprise_vectors", snapshot_name="snapshot_file.snapshot")Deletes snapshot file from storage.
41Create Full System Snapshotclient.create_full_snapshot()Takes snapshot of all collections and cluster state.
42List Full System Snapshotsclient.list_full_snapshots()Lists all system-wide snapshots.
43Delete Full System Snapshotclient.delete_full_snapshot(snapshot_name="full_snap.snapshot")Deletes system-wide snapshot.
44Create Integer Payload Indexclient.create_payload_index(collection_name="enterprise_vectors", field_name="age", field_schema=models.PayloadSchemaType.INTEGER)Provisions integer filter index.
45Create Float Payload Indexclient.create_payload_index(collection_name="enterprise_vectors", field_name="score", field_schema=models.PayloadSchemaType.FLOAT)Provisions float filter index.
46Create Geo Payload Indexclient.create_payload_index(collection_name="enterprise_vectors", field_name="loc", field_schema=models.PayloadSchemaType.GEO)Provisions geospatial filter index.
47Create Text Payload Indexclient.create_payload_index(collection_name="enterprise_vectors", field_name="description", field_schema=models.TextIndexParams(type="text", tokenizer=models.TokenizerType.WORD))Provisions fulltext payload filter index.
48Search Geo Bounding Boxclient.search(collection_name="enterprise_vectors", query_vector=[0.1, ...], query_filter=models.Filter(must=[models.FieldCondition(key="loc", geo_bounding_box=models.GeoBoundingBox(top_left=models.GeoPoint(lon=-74.0, lat=40.8), bottom_right=models.GeoPoint(lon=-73.8, lat=40.7)))]))Filters search results within spatial bounding box.
49Search Geo Radiusclient.search(collection_name="enterprise_vectors", query_vector=[0.1, ...], query_filter=models.Filter(must=[models.FieldCondition(key="loc", geo_radius=models.GeoRadius(center=models.GeoPoint(lon=-73.9, lat=40.7), radius=5000.0))]))Filters search results within 5km radius.
50Search Range Filterclient.search(collection_name="enterprise_vectors", query_vector=[0.1, ...], query_filter=models.Filter(must=[models.FieldCondition(key="age", range=models.Range(gte=18, lte=65))]))Filters search results within numeric range.
51Search Match Any Filterclient.search(collection_name="enterprise_vectors", query_vector=[0.1, ...], query_filter=models.Filter(must=[models.FieldCondition(key="category", match=models.MatchAny(any=["tech", "finance"]))]))Filters search results matching array of values.
52Search Is Null Filterclient.search(collection_name="enterprise_vectors", query_vector=[0.1, ...], query_filter=models.Filter(must_not=[models.IsEmptyCondition(is_empty=models.PayloadField(key="category"))]))Filters out vectors where payload field is empty.
53Delete Payload Fieldclient.delete_payload(collection_name="enterprise_vectors", keys=["status"], points=[1])Removes specific keys from metadata payload.
54Overwrite Payloadclient.overwrite_payload(collection_name="enterprise_vectors", payload={"role": "admin"}, points=[1])Replaces entire metadata payload for point.
55Check Sharding Optimizer Configclient.update_collection(collection_name="enterprise_vectors", optimizer_config=models.OptimizersConfigDiff(indexing_threshold=20000))Configures HNSW indexing threshold.
56Check HNSW Config Updateclient.update_collection(collection_name="enterprise_vectors", hnsw_config=models.HnswConfigDiff(m=32, ef_construct=200))Updates graph HNSW parameters on live collection.
57Enable Product Quantizationclient.update_collection(collection_name="enterprise_vectors", quantization_config=models.ProductQuantization(product=models.ProductQuantizationConfig(compression=models.CompressionRatio.X16)))Enforces 16x product quantization for memory reduction.
58Disable Quantizationclient.update_collection(collection_name="enterprise_vectors", quantization_config=models.Disabled())Disables vector quantization.
59Update Collection Paramsclient.update_collection(collection_name="enterprise_vectors", params=models.CollectionParamsDiff(replication_factor=2))Updates replication factor across cluster.
60Check Cluster Peer Listclient.get_cluster_info()Inspects active cluster peer nodes and consensus state.
61Recover Shard Peerclient.recover_shard(collection_name="enterprise_vectors", shard_id=0, peer_id=1)Triggers shard recovery on peer node.
62Drop Shard Replicaclient.delete_shard_key(collection_name="enterprise_vectors", shard_key="tenant_a")Deletes custom shard key partition.
63Create Custom Shard Keyclient.create_shard_key(collection_name="enterprise_vectors", shard_key="tenant_a")Provisions custom shard key for tenant isolation.
64Upsert Point with Shard Keyclient.upsert(collection_name="enterprise_vectors", points=[...], shard_key_selector="tenant_a")Writes vector point to custom tenant shard.
65Search Point with Shard Keyclient.search(collection_name="enterprise_vectors", query_vector=[0.1, ...], shard_key_selector="tenant_a")Searches vectors within custom tenant shard.
66Count Points with Shard Keyclient.count(collection_name="enterprise_vectors", shard_key_selector="tenant_a")Calculates point count in tenant shard.
67Scroll Points with Shard Keyclient.scroll(collection_name="enterprise_vectors", shard_key_selector="tenant_a")Paginates records in tenant shard.
68Delete Points with Shard Keyclient.delete(collection_name="enterprise_vectors", points_selector=..., shard_key_selector="tenant_a")Deletes records in tenant shard.
69Search Groups (Field Grouping)client.search_groups(collection_name="enterprise_vectors", query_vector=[0.1, ...], group_by="document_id", group_size=3, limit=10)Groups vector search results by document ID.
70Recommend Groupsclient.recommend_groups(collection_name="enterprise_vectors", positive=[1], group_by="document_id", group_size=3, limit=10)Groups recommendation results by document ID.
71Query Points Universal APIclient.query_points(collection_name="enterprise_vectors", query=[0.1, ...], limit=5)Executes universal query API in Qdrant 1.7+.
72Query Points Nearest Neighborclient.query_points(collection_name="enterprise_vectors", query=models.NearestQuery(nearest=[0.1, ...]), limit=5)Queries nearest neighbors via Universal API.
73Query Points Fusion RRFclient.query_points(collection_name="enterprise_vectors", query=models.FusionQuery(fusion=models.Fusion.RRF), limit=5)Executes Reciprocal Rank Fusion on search results.
74Query Points Sampleclient.query_points(collection_name="enterprise_vectors", query=models.SampleQuery(sample=models.Sample.RANDOM), limit=5)Returns random sample of points.
75Check Server Lock Statusclient.get_locks()Inspects current write locks on server.
76Set Server Write Lockclient.post_locks(has_write_locks=True, error_message="Maintenance mode")Enforces global read-only lock.
77Clear Server Write Lockclient.post_locks(has_write_locks=False)Clears global read-only lock.
78Check Health Probe REST APIcurl http://localhost:6333/healthzQueries HTTP REST endpoint for server health.
79Check Readiness Probe REST APIcurl http://localhost:6333/readyzQueries HTTP REST endpoint for server readiness probe.
80Check OpenAPI Schema REST APIcurl http://localhost:6333/api-dataFetches JSON OpenAPI specification schema.
81Check Telemetry REST APIcurl http://localhost:6333/telemetryPulls server telemetry diagnostics.
82Check Qdrant Client Versionimport qdrant_client; print(qdrant_client.__version__)Outputs Qdrant Python client library version string.
83Verify API Key AuthenticationQdrantClient(url="http://localhost:6333", api_key="pc_secure_key")Authenticates Qdrant client via API key header.
84Set Client TimeoutQdrantClient(url="http://localhost:6333", timeout=30.0)Sets 30-second gRPC/REST socket timeout.
85Close gRPC Client Channelsclient.close()Gracefully closes gRPC network channels to Qdrant server.
86Check Collection Optimizer Statusclient.get_collection(collection_name='enterprise_vectors').optimizer_statusInspects collection background optimizer status.
87Check Collection Payload Schemaclient.get_collection(collection_name='enterprise_vectors').payload_schemaInspects created payload indexes and schemas.
88Check Collection Vectors Configclient.get_collection(collection_name='enterprise_vectors').config.params.vectorsInspects vector dimension and metric configuration.
89Check Collection Quantization Statusclient.get_collection(collection_name='enterprise_vectors').config.quantization_configInspects quantization settings on collection.
90Check Collection HNSW Configclient.get_collection(collection_name='enterprise_vectors').config.hnsw_configInspects HNSW graph construction parameters.
91Check Collection Points Count Exactclient.count(collection_name='enterprise_vectors', exact=True).countCalculates exact point count in collection.
92Check Collection Points Count Fastclient.count(collection_name='enterprise_vectors', exact=False).countFetches fast approximate point count.
93Check Cluster Peer Consensus Infoclient.get_cluster_info().peer_idExtracts local peer node ID in cluster.
94Check Cluster Peer Consensus Stateclient.get_cluster_info().stateInspects consensus state (Leader/Follower).
95Check Cluster Total Peers Countlen(client.get_cluster_info().peers)Calculates total peer nodes in cluster.
96Check Collection Aliases Countlen(client.get_collection_aliases(collection_name='enterprise_vectors').aliases)Calculates count of active aliases for collection.
97Check Snapshots Countlen(client.list_snapshots(collection_name='enterprise_vectors'))Calculates total saved snapshots for collection.
98Check Full System Snapshots Countlen(client.list_full_snapshots())Calculates total saved system-wide snapshots.
99Check Client User Agent Headerclient.openapi_client.client.user_agentInspects client HTTP user agent header string.
100Verify Qdrant REST Server Healthcurl -s http://localhost:6333/healthzQueries HTTP REST endpoint for server health.
101Connect to Qdrant Client Option 1from qdrant_client import QdrantClient; client = QdrantClient(url="http://localhost:6333")Connects to Qdrant vector database.
102Check Server Health Option 2client.get_telemetry()Verifies Qdrant server health state.
103Create Collection Option 3from qdrant_client.http import models; client.create_collection(collection_name="enterprise_vectors", vectors_config=models.VectorParams(size=1536, distance=models.Distance.COSINE))Provisions vector collection.
104Delete Collection Option 4client.delete_collection(collection_name="enterprise_vectors")Deletes vector collection.
105List Collections Option 5client.get_collections()Outputs all vector collections.
106Get Collection Info Option 6client.get_collection(collection_name="enterprise_vectors")Inspects vector count and index status.
107Upsert Vectors Option 7client.upsert(collection_name="enterprise_vectors", points=[models.PointStruct(id=1, vector=[0.1, ...], payload={"category": "tech"})])Writes vector embeddings and metadata payload.
108Retrieve Points by ID Option 8client.retrieve(collection_name="enterprise_vectors", ids=[1])Fetches vectors by ID.
109Delete Points Option 9client.delete(collection_name="enterprise_vectors", points_selector=models.PointIdsList(points=[1]))Removes vector points.
110Execute Vector Similarity Search Option 10client.search(collection_name="enterprise_vectors", query_vector=[0.1, ...], limit=5)Executes nearest-neighbor vector search.
111Search with Payload Filter Option 11client.search(collection_name="enterprise_vectors", query_vector=[0.1, ...], query_filter=models.Filter(must=[models.FieldCondition(key="category", match=models.MatchValue(value="tech"))]), limit=5)Executes filtered vector search.
112Create Payload Index Option 12client.create_payload_index(collection_name="enterprise_vectors", field_name="category", field_schema=models.PayloadSchemaType.KEYWORD)Provisions metadata filter index.
113Delete Payload Index Option 13client.delete_payload_index(collection_name="enterprise_vectors", field_name="category")Removes metadata filter index.
114Create Snapshot Option 14client.create_snapshot(collection_name="enterprise_vectors")Takes collection storage snapshot.
115List Snapshots Option 15client.list_snapshots(collection_name="enterprise_vectors")Outputs all collection snapshots.
116Recover from Snapshot Option 16client.recover_snapshot(collection_name="enterprise_vectors", snapshot_name="snapshot_file.snapshot")Restores collection from snapshot.
117Create Quantization (Compression) Option 17client.update_collection(collection_name="enterprise_vectors", quantization_config=models.ScalarQuantization(scalar=models.ScalarQuantizationConfig(type=models.ScalarType.INT8)))Enforces scalar vector quantization for memory reduction.
118Create Alias Option 18client.update_collection_aliases(actions=[models.CreateAliasOperation(alias_name="prod_vectors", collection_name="enterprise_vectors")])Provisions alias pointer for collection.
119Delete Alias Option 19client.update_collection_aliases(actions=[models.DeleteAliasOperation(alias_name="prod_vectors")])Removes collection alias.
120List Aliases Option 20client.get_collection_aliases(collection_name="enterprise_vectors")Outputs aliases for collection.
121Scroll Through Points Option 21client.scroll(collection_name="enterprise_vectors", limit=100)Paginates through vector collection records.
122Count Points Option 22client.count(collection_name="enterprise_vectors", exact=True)Calculates exact vector point count.
123Recommend Vectors Option 23client.recommend(collection_name="enterprise_vectors", positive=[1], negative=[2], limit=5)Recommends vectors based on positive/negative examples.
124Discover Vectors Option 24client.discover(collection_name="enterprise_vectors", target=[0.1, ...], context=[models.ContextExample(positive=[1], negative=[2])])Discovers vectors based on contextual intent.
125Cluster Points Option 25client.cluster(collection_name="enterprise_vectors")Inspects distributed cluster shard topology.
126Update Point Payload Option 26client.set_payload(collection_name="enterprise_vectors", payload={"status": "active"}, points=[1])Modifies metadata payload.
127Clear Payload Option 27client.clear_payload(collection_name="enterprise_vectors", points_selector=models.PointIdsList(points=[1]))Removes metadata payload fields.
128Export Metrics (Prometheus) Option 28curl http://localhost:6333/metricsPulls Qdrant server telemetry metrics.
129Check Cluster Consensus Option 29client.print_locks()Inspects distributed consensus raft locks.
130Close Client Connection Option 30client.close()Closes connection to Qdrant server.
131Check Has Collection Option 31client.collection_exists(collection_name="enterprise_vectors")Verifies whether vector collection exists.
132Create Multi-Vector Collection Option 32client.create_collection(collection_name="multi_vectors", vectors_config={"text": models.VectorParams(size=1536, distance=models.Distance.COSINE), "image": models.VectorParams(size=512, distance=models.Distance.EUCLID)})Provisions collection with multiple named vector fields.
133Upsert Multi-Vectors Option 33client.upsert(collection_name="multi_vectors", points=[models.PointStruct(id=1, vector={"text": [0.1, ...], "image": [0.2, ...]})])Writes named multi-vector point.
134Search Named Vector Option 34client.search(collection_name="multi_vectors", query_vector=("text", [0.1, ...]), limit=5)Executes search on specific named vector field.
135Create Sparse Vector Collection Option 35client.create_collection(collection_name="sparse_vectors", sparse_vectors_config={"bm25": models.SparseVectorParams()})Provisions collection supporting sparse vectors.
136Upsert Sparse Vectors Option 36client.upsert(collection_name="sparse_vectors", points=[models.PointStruct(id=1, vector={"bm25": models.SparseVector(indices=[1, 12, 45], values=[0.1, 0.8, 0.3])})])Writes sparse vector keyword embeddings.
137Search Sparse Vector Option 37client.search(collection_name="sparse_vectors", query_vector=models.NamedSparseVector(name="bm25", vector=models.SparseVector(indices=[1], values=[1.0])), limit=5)Executes sparse vector search query.
138Query Search Batch Option 38client.search_batch(collection_name="enterprise_vectors", requests=[models.SearchRequest(vector=[0.1, ...], limit=5), models.SearchRequest(vector=[0.2, ...], limit=5)])Executes multiple vector searches in single RPC roundtrip.
139Recommend Batch Option 39client.recommend_batch(collection_name="enterprise_vectors", requests=[models.RecommendRequest(positive=[1], limit=5)])Executes multiple recommendation queries in batch.
140Delete Snapshot File Option 40client.delete_snapshot(collection_name="enterprise_vectors", snapshot_name="snapshot_file.snapshot")Deletes snapshot file from storage.
141Create Full System Snapshot Option 41client.create_full_snapshot()Takes snapshot of all collections and cluster state.
142List Full System Snapshots Option 42client.list_full_snapshots()Lists all system-wide snapshots.
143Delete Full System Snapshot Option 43client.delete_full_snapshot(snapshot_name="full_snap.snapshot")Deletes system-wide snapshot.
144Create Integer Payload Index Option 44client.create_payload_index(collection_name="enterprise_vectors", field_name="age", field_schema=models.PayloadSchemaType.INTEGER)Provisions integer filter index.
145Create Float Payload Index Option 45client.create_payload_index(collection_name="enterprise_vectors", field_name="score", field_schema=models.PayloadSchemaType.FLOAT)Provisions float filter index.
146Create Geo Payload Index Option 46client.create_payload_index(collection_name="enterprise_vectors", field_name="loc", field_schema=models.PayloadSchemaType.GEO)Provisions geospatial filter index.
147Create Text Payload Index Option 47client.create_payload_index(collection_name="enterprise_vectors", field_name="description", field_schema=models.TextIndexParams(type="text", tokenizer=models.TokenizerType.WORD))Provisions fulltext payload filter index.
148Search Geo Bounding Box Option 48client.search(collection_name="enterprise_vectors", query_vector=[0.1, ...], query_filter=models.Filter(must=[models.FieldCondition(key="loc", geo_bounding_box=models.GeoBoundingBox(top_left=models.GeoPoint(lon=-74.0, lat=40.8), bottom_right=models.GeoPoint(lon=-73.8, lat=40.7)))]))Filters search results within spatial bounding box.
149Search Geo Radius Option 49client.search(collection_name="enterprise_vectors", query_vector=[0.1, ...], query_filter=models.Filter(must=[models.FieldCondition(key="loc", geo_radius=models.GeoRadius(center=models.GeoPoint(lon=-73.9, lat=40.7), radius=5000.0))]))Filters search results within 5km radius.
150Search Range Filter Option 50client.search(collection_name="enterprise_vectors", query_vector=[0.1, ...], query_filter=models.Filter(must=[models.FieldCondition(key="age", range=models.Range(gte=18, lte=65))]))Filters search results within numeric range.
151Search Match Any Filter Option 51client.search(collection_name="enterprise_vectors", query_vector=[0.1, ...], query_filter=models.Filter(must=[models.FieldCondition(key="category", match=models.MatchAny(any=["tech", "finance"]))]))Filters search results matching array of values.
152Search Is Null Filter Option 52client.search(collection_name="enterprise_vectors", query_vector=[0.1, ...], query_filter=models.Filter(must_not=[models.IsEmptyCondition(is_empty=models.PayloadField(key="category"))]))Filters out vectors where payload field is empty.
153Delete Payload Field Option 53client.delete_payload(collection_name="enterprise_vectors", keys=["status"], points=[1])Removes specific keys from metadata payload.
154Overwrite Payload Option 54client.overwrite_payload(collection_name="enterprise_vectors", payload={"role": "admin"}, points=[1])Replaces entire metadata payload for point.
155Check Sharding Optimizer Config Option 55client.update_collection(collection_name="enterprise_vectors", optimizer_config=models.OptimizersConfigDiff(indexing_threshold=20000))Configures HNSW indexing threshold.
156Check HNSW Config Update Option 56client.update_collection(collection_name="enterprise_vectors", hnsw_config=models.HnswConfigDiff(m=32, ef_construct=200))Updates graph HNSW parameters on live collection.
157Enable Product Quantization Option 57client.update_collection(collection_name="enterprise_vectors", quantization_config=models.ProductQuantization(product=models.ProductQuantizationConfig(compression=models.CompressionRatio.X16)))Enforces 16x product quantization for memory reduction.
158Disable Quantization Option 58client.update_collection(collection_name="enterprise_vectors", quantization_config=models.Disabled())Disables vector quantization.
159Update Collection Params Option 59client.update_collection(collection_name="enterprise_vectors", params=models.CollectionParamsDiff(replication_factor=2))Updates replication factor across cluster.
160Check Cluster Peer List Option 60client.get_cluster_info()Inspects active cluster peer nodes and consensus state.
161Recover Shard Peer Option 61client.recover_shard(collection_name="enterprise_vectors", shard_id=0, peer_id=1)Triggers shard recovery on peer node.
162Drop Shard Replica Option 62client.delete_shard_key(collection_name="enterprise_vectors", shard_key="tenant_a")Deletes custom shard key partition.
163Create Custom Shard Key Option 63client.create_shard_key(collection_name="enterprise_vectors", shard_key="tenant_a")Provisions custom shard key for tenant isolation.
164Upsert Point with Shard Key Option 64client.upsert(collection_name="enterprise_vectors", points=[...], shard_key_selector="tenant_a")Writes vector point to custom tenant shard.
165Search Point with Shard Key Option 65client.search(collection_name="enterprise_vectors", query_vector=[0.1, ...], shard_key_selector="tenant_a")Searches vectors within custom tenant shard.
166Count Points with Shard Key Option 66client.count(collection_name="enterprise_vectors", shard_key_selector="tenant_a")Calculates point count in tenant shard.
167Scroll Points with Shard Key Option 67client.scroll(collection_name="enterprise_vectors", shard_key_selector="tenant_a")Paginates records in tenant shard.
168Delete Points with Shard Key Option 68client.delete(collection_name="enterprise_vectors", points_selector=..., shard_key_selector="tenant_a")Deletes records in tenant shard.
169Search Groups (Field Grouping) Option 69client.search_groups(collection_name="enterprise_vectors", query_vector=[0.1, ...], group_by="document_id", group_size=3, limit=10)Groups vector search results by document ID.
170Recommend Groups Option 70client.recommend_groups(collection_name="enterprise_vectors", positive=[1], group_by="document_id", group_size=3, limit=10)Groups recommendation results by document ID.
171Query Points Universal API Option 71client.query_points(collection_name="enterprise_vectors", query=[0.1, ...], limit=5)Executes universal query API in Qdrant 1.7+.
172Query Points Nearest Neighbor Option 72client.query_points(collection_name="enterprise_vectors", query=models.NearestQuery(nearest=[0.1, ...]), limit=5)Queries nearest neighbors via Universal API.
173Query Points Fusion RRF Option 73client.query_points(collection_name="enterprise_vectors", query=models.FusionQuery(fusion=models.Fusion.RRF), limit=5)Executes Reciprocal Rank Fusion on search results.
174Query Points Sample Option 74client.query_points(collection_name="enterprise_vectors", query=models.SampleQuery(sample=models.Sample.RANDOM), limit=5)Returns random sample of points.
175Check Server Lock Status Option 75client.get_locks()Inspects current write locks on server.
176Set Server Write Lock Option 76client.post_locks(has_write_locks=True, error_message="Maintenance mode")Enforces global read-only lock.
177Clear Server Write Lock Option 77client.post_locks(has_write_locks=False)Clears global read-only lock.
178Check Health Probe REST API Option 78curl http://localhost:6333/healthzQueries HTTP REST endpoint for server health.
179Check Readiness Probe REST API Option 79curl http://localhost:6333/readyzQueries HTTP REST endpoint for server readiness probe.
180Check OpenAPI Schema REST API Option 80curl http://localhost:6333/api-dataFetches JSON OpenAPI specification schema.
181Check Telemetry REST API Option 81curl http://localhost:6333/telemetryPulls server telemetry diagnostics.
182Check Qdrant Client Version Option 82import qdrant_client; print(qdrant_client.__version__)Outputs Qdrant Python client library version string.
183Verify API Key Authentication Option 83QdrantClient(url="http://localhost:6333", api_key="pc_secure_key")Authenticates Qdrant client via API key header.
184Set Client Timeout Option 84QdrantClient(url="http://localhost:6333", timeout=30.0)Sets 30-second gRPC/REST socket timeout.
185Close gRPC Client Channels Option 85client.close()Gracefully closes gRPC network channels to Qdrant server.
186Check Collection Optimizer Status Option 86client.get_collection(collection_name='enterprise_vectors').optimizer_statusInspects collection background optimizer status.
187Check Collection Payload Schema Option 87client.get_collection(collection_name='enterprise_vectors').payload_schemaInspects created payload indexes and schemas.
188Check Collection Vectors Config Option 88client.get_collection(collection_name='enterprise_vectors').config.params.vectorsInspects vector dimension and metric configuration.
189Check Collection Quantization Status Option 89client.get_collection(collection_name='enterprise_vectors').config.quantization_configInspects quantization settings on collection.
190Check Collection HNSW Config Option 90client.get_collection(collection_name='enterprise_vectors').config.hnsw_configInspects HNSW graph construction parameters.
191Check Collection Points Count Exact Option 91client.count(collection_name='enterprise_vectors', exact=True).countCalculates exact point count in collection.
192Check Collection Points Count Fast Option 92client.count(collection_name='enterprise_vectors', exact=False).countFetches fast approximate point count.
193Check Cluster Peer Consensus Info Option 93client.get_cluster_info().peer_idExtracts local peer node ID in cluster.
194Check Cluster Peer Consensus State Option 94client.get_cluster_info().stateInspects consensus state (Leader/Follower).
195Check Cluster Total Peers Count Option 95len(client.get_cluster_info().peers)Calculates total peer nodes in cluster.
196Check Collection Aliases Count Option 96len(client.get_collection_aliases(collection_name='enterprise_vectors').aliases)Calculates count of active aliases for collection.
197Check Snapshots Count Option 97len(client.list_snapshots(collection_name='enterprise_vectors'))Calculates total saved snapshots for collection.
198Check Full System Snapshots Count Option 98len(client.list_full_snapshots())Calculates total saved system-wide snapshots.
199Check Client User Agent Header Option 99client.openapi_client.client.user_agentInspects client HTTP user agent header string.
200Verify Qdrant REST Server Health Option 100curl -s http://localhost:6333/healthzQueries HTTP REST endpoint for server health.

Pinecone Vector Index VECTOR

Technical Architecture & Overview

Pinecone Vector Index is a fully managed, serverless vector database designed for high-performance AI vector search without infrastructure overhead. It automatically handles index creation, scaling, sparse-dense hybrid vector indexing, namespace data isolation, and low-latency nearest-neighbor search across cloud regions.

Primary Use Cases: Serverless RAG pipelines, enterprise AI search engines, multi-tenant vector storage, semantic document retrieval, and real-time personalized recommendations.

Core Components: Serverless Index Fabric, Sparse-Dense Vector Indexer, Namespace Partitioning Engine, Metadata Filtering Engine, and gRPC / REST Python SDK.

Exhaustive 200-command administrative, diagnostic, and operational matrix for Pinecone Vector Index.

#Operation / ActionPinecone Python SDK SyntaxDescription
1Initialize Pinecone Clientfrom pinecone import Pinecone; pc = Pinecone(api_key="pc_secure_key")Authenticates Pinecone SDK session.
2Create Serverless Indexpc.create_index(name="enterprise-index", dimension=1536, metric="cosine", spec=ServerlessSpec(cloud="aws", region="us-east-1"))Provisions managed serverless vector index.
3Delete Indexpc.delete_index(name="enterprise-index")Deletes serverless vector index.
4List Indexespc.list_indexes()Outputs all vector indexes.
5Describe Indexpc.describe_index(name="enterprise-index")Inspects vector index dimensions and status.
6Connect to Index Hostindex = pc.Index("enterprise-index")Connects to specific vector index instance.
7Upsert Vectorsindex.upsert(vectors=[{"id": "vec1", "values": [0.1, ...], "metadata": {"category": "tech"}}])Writes vector embeddings and metadata payload.
8Query Vectorsindex.query(vector=[0.1, ...], top_k=5, include_metadata=True)Executes nearest-neighbor vector similarity search.
9Query with Metadata Filterindex.query(vector=[0.1, ...], filter={"category": {"$eq": "tech"}}, top_k=5, include_metadata=True)Executes filtered vector search.
10Fetch Vectors by IDindex.fetch(ids=["vec1"])Retrieves vectors by record ID.
11Delete Vectors by IDindex.delete(ids=["vec1"])Removes vector records.
12Delete Vectors by Filterindex.delete(filter={"category": {"$eq": "tech"}})Removes vectors matching metadata filter.
13Delete All Vectors (Clear)index.delete(delete_all=true)Clears all vectors from index.
14Check Index Statisticsindex.describe_index_stats()Calculates vector counts and namespace distribution.
15Create Collection (Backup)pc.create_collection(name="index-backup", source_index="enterprise-index")Takes serverless index snapshot collection.
16List Collectionspc.list_collections()Outputs all index backup collections.
17Delete Collectionpc.delete_collection(name="index-backup")Removes index backup collection.
18Create Pod-Based Index (GPU/CPU)pc.create_index(name="pod-index", dimension=1536, metric="cosine", spec=PodSpec(environment="us-west1-gcp", pod_type="p1.x1", pods=1))Provisions dedicated pod vector index.
19Configure Pod Index Scalepc.configure_index(name="pod-index", replicas=2, pod_type="p1.x2")Scales pod compute resources.
20Create Namespaceindex.upsert(vectors=[...], namespace="tenant-a")Provisions isolated tenant namespace.
21Query Specific Namespaceindex.query(vector=[0.1, ...], namespace="tenant-a", top_k=5)Executes vector search within tenant namespace.
22Update Vector Recordindex.update(id="vec1", values=[0.2, ...], set_metadata={"status": "updated"})Modifies existing vector values and metadata.
23Start Import Job (S3)index.start_import(uri="s3://my-bucket/vectors/")Provisions bulk vector import from cloud storage.
24List Import Jobsindex.list_imports()Outputs bulk import job statuses.
25Cancel Import Jobindex.cancel_import(import_id="import-123")Stops active bulk vector import job.
26Check Import Job Statusindex.describe_import(import_id="import-123")Inspects vector import job progress.
27Create API Keypc.create_api_key(name="app-key")Provisions programmatic access token.
28List API Keyspc.list_api_keys()Outputs active API keys.
29Delete API Keypc.delete_api_key(api_key_id="key-123")Revokes API access token.
30Check Pinecone SDK Versionimport pinecone; print(pinecone.__version__)Outputs running Pinecone Python SDK version.
31Check Has Indexpc.has_index(name="enterprise-index")Verifies whether vector index exists in account.
32Describe Collection Detailspc.describe_collection(name="index-backup")Inspects size, record count, and source index of collection snapshot.
33Upsert Vectors with Namespaceindex.upsert(vectors=[...], namespace="tenant-b")Writes vectors into tenant-b isolated namespace.
34Query Vectors Include Valuesindex.query(vector=[0.1, ...], top_k=5, include_values=True)Returns raw float vector array in query response.
35Query ID Searchindex.query(id="vec1", top_k=5, include_metadata=True)Executes vector search using vector ID as query vector.
36Query Filter In Operatorindex.query(vector=[0.1, ...], filter={"category": {"$in": ["tech", "finance"]}}, top_k=5)Filters search matching array of values.
37Query Filter Not In Operatorindex.query(vector=[0.1, ...], filter={"category": {"$nin": ["archived"]}}, top_k=5)Filters search excluding specified values.
38Query Filter Greater Than Operatorindex.query(vector=[0.1, ...], filter={"price": {"$gt": 100}}, top_k=5)Filters search where numeric property > 100.
39Query Filter Less Than Operatorindex.query(vector=[0.1, ...], filter={"price": {"$lte": 500}}, top_k=5)Filters search where numeric property <= 500.
40Query Filter And Operatorindex.query(vector=[0.1, ...], filter={"$and": [{"category": {"$eq": "tech"}}, {"price": {"$lt": 200}}]}, top_k=5)Filters search matching AND conditions.
41Query Filter Or Operatorindex.query(vector=[0.1, ...], filter={"$or": [{"category": {"$eq": "tech"}}, {"featured": {"$eq": true}}]}, top_k=5)Filters search matching OR conditions.
42Fetch Multiple Vectorsindex.fetch(ids=["vec1", "vec2", "vec3"], namespace="tenant-a")Batch fetches vectors by IDs from namespace.
43Delete Vectors in Namespaceindex.delete(delete_all=True, namespace="tenant-a")Purges all vectors from specific namespace.
44Update Vector Metadata Onlyindex.update(id="vec1", set_metadata={"status": "verified"}, namespace="tenant-a")Updates metadata attributes without rewriting vector array.
45Update Vector Values Onlyindex.update(id="vec1", values=[0.3, ...], namespace="tenant-a")Updates vector float array without changing metadata.
46List Vectors in Namespace (Paginated)for ids_batch in index.list(namespace="tenant-a"): print(ids_batch)Paginates through vector IDs in namespace.
47List Vectors with Prefix Filterindex.list(prefix="user_", namespace="tenant-a")Lists vector IDs matching prefix string.
48Create Sparse-Dense Hybrid Indexpc.create_index(name="hybrid-index", dimension=1536, metric="dotproduct", spec=ServerlessSpec(cloud="aws", region="us-east-1"))Provisions index supporting sparse-dense hybrid vectors.
49Upsert Sparse-Dense Vectorindex.upsert(vectors=[{"id": "vec1", "values": [0.1, ...], "sparse_values": {"indices": [1, 5], "values": [0.8, 0.4]}}])Writes combined sparse-dense vector embedding.
50Query Sparse-Dense Hybrid Searchindex.query(vector=[0.1, ...], sparse_vector=SparseValues(indices=[1, 5], values=[0.8, 0.4]), top_k=5)Executes hybrid vector similarity search.
51Configure Index Replicaspc.configure_index(name="pod-index", replicas=3)Scales read replica pods across availability zones.
52Configure Index Pod Typepc.configure_index(name="pod-index", pod_type="p2.x1")Scales up compute pod hardware type.
53Create Serverless Index GCPpc.create_index(name="gcp-index", dimension=1536, metric="cosine", spec=ServerlessSpec(cloud="gcp", region="us-central1"))Provisions serverless index in GCP region.
54Create Serverless Index Azurepc.create_index(name="azure-index", dimension=1536, metric="cosine", spec=ServerlessSpec(cloud="azure", region="eastus2"))Provisions serverless index in Azure region.
55Check Index Host URLindex_info = pc.describe_index(name="enterprise-index"); print(index_info.host)Extracts gRPC/REST endpoint host URL.
56Connect Index via Direct Host URLindex = pc.Index(host="https://enterprise-index-xyz.svc.us-east-1.pinecone.io")Connects to index using explicit endpoint URL.
57Check Index Deletion Protectionpc.configure_index(name="enterprise-index", deletion_protection="enabled")Enforces deletion protection on index.
58Disable Index Deletion Protectionpc.configure_index(name="enterprise-index", deletion_protection="disabled")Disables deletion protection on index.
59Create Index from Collection Backuppc.create_index(name="restored-index", dimension=1536, metric="cosine", spec=PodSpec(environment="us-west1-gcp", pod_type="p1.x1", source_collection="index-backup"))Restores vector index from collection snapshot.
60Check Collection Statuscollection_info = pc.describe_collection(name="index-backup"); print(collection_info.status)Verifies if collection snapshot status is Ready.
61Check Collection Size Bytescollection_info = pc.describe_collection(name="index-backup"); print(collection_info.size)Calculates physical storage bytes of collection snapshot.
62Check Index Vector Count Statsstats = index.describe_index_stats(); print(stats.total_vector_count)Calculates total vectors across all namespaces.
63Check Index Dimension Statsstats = index.describe_index_stats(); print(stats.dimension)Inspects vector dimension size.
64Check Index Namespaces Liststats = index.describe_index_stats(); print(stats.namespaces)Lists all active namespaces and vector counts.
65Check Index Fullness Metricstats = index.describe_index_stats(); print(stats.index_fullness)Inspects pod capacity utilization percentage.
66Import Vectors Integration S3 Roleindex.start_import(uri="s3://my-bucket/vectors/", integration_id="int-123")Imports vectors using AWS IAM integration role.
67Check Import Errorsjob = index.describe_import(import_id="import-123"); print(job.error)Inspects bulk import error log.
68Check Import Records Processedjob = index.describe_import(import_id="import-123"); print(job.records_imported)Inspects count of successfully imported vector records.
69Create Integration AWS IAMpc.create_integration(name="s3-int", type="s3", aws_role_arn="arn:aws:iam::123:role/PineconeRole")Provisions cloud storage integration.
70List Integrationspc.list_integrations()Lists registered cloud storage integrations.
71Describe Integrationpc.describe_integration(name="s3-int")Inspects status of cloud storage integration.
72Delete Integrationpc.delete_integration(name="s3-int")Deletes cloud storage integration.
73Check API Key Permissionspc.describe_api_key(api_key_id="key-123")Inspects scope and creation timestamp of API key.
74Check Account Metrics CloudWatchaws cloudwatch get-metric-data --metric-data-queries ...Pulls Pinecone vector search latency and QPS from CloudWatch.
75Check Index REST Search HTTPcurl -X POST -H 'Api-Key: pc_secure_key' -H 'Content-Type: application/json' -d '{"vector": [0.1, ...], "topK": 5}' https://enterprise-index-xyz.svc.us-east-1.pinecone.io/queryExecutes REST API search request directly.
76Check Index REST Upsert HTTPcurl -X POST -H 'Api-Key: pc_secure_key' -H 'Content-Type: application/json' -d '{"vectors": [{"id": "vec1", "values": [0.1, ...]}]}' https://enterprise-index-xyz.svc.us-east-1.pinecone.io/vectors/upsertExecutes REST API upsert request directly.
77Check Index REST Fetch HTTPcurl -X GET -H 'Api-Key: pc_secure_key' 'https://enterprise-index-xyz.svc.us-east-1.pinecone.io/vectors/fetch?ids=vec1'Executes REST API fetch request directly.
78Check Index REST Delete HTTPcurl -X POST -H 'Api-Key: pc_secure_key' -H 'Content-Type: application/json' -d '{"ids": ["vec1"]}' https://enterprise-index-xyz.svc.us-east-1.pinecone.io/vectors/deleteExecutes REST API delete request directly.
79Check Index REST Describe Stats HTTPcurl -X POST -H 'Api-Key: pc_secure_key' https://enterprise-index-xyz.svc.us-east-1.pinecone.io/describe_index_statsExecutes REST API describe index stats request.
80Verify Pinecone gRPC Channel SSLindex = pc.Index("enterprise-index", pool_threads=30)Provisions connection pool with 30 concurrent threads.
81Check Client User Agent Stringpc = Pinecone(api_key="pc_secure_key", source_tag="custom_app")Sets custom source tag in client user agent header.
82Close Pinecone Client Connectionindex.close()Closes active gRPC network channels to Pinecone index.
83Check Index Name List[idx.name for idx in pc.list_indexes()]Extracts list of index names.
84Check Collection Name List[col.name for idx in pc.list_collections()]Extracts list of collection snapshot names.
85Check Index Dimensionpc.describe_index(name='enterprise-index').dimensionInspects vector dimension size.
86Check Index Metricpc.describe_index(name='enterprise-index').metricInspects vector distance metric (cosine/dotproduct/euclidean).
87Check Index Host Endpointpc.describe_index(name='enterprise-index').hostExtracts index host endpoint URL.
88Check Index Status Readypc.describe_index(name='enterprise-index').status.readyVerifies if index status is ready.
89Check Index Status Statepc.describe_index(name='enterprise-index').status.stateInspects index operational state string.
90Check Index Spec Cloudpc.describe_index(name='enterprise-index').spec.serverless.cloudInspects serverless cloud provider (aws/gcp/azure).
91Check Index Spec Regionpc.describe_index(name='enterprise-index').spec.serverless.regionInspects serverless cloud region.
92Check Index Stats Total Vector Countindex.describe_index_stats().total_vector_countCalculates total vector count across index.
93Check Index Stats Namespaces Dictindex.describe_index_stats().namespacesPulls dictionary of namespaces and vector counts.
94Check Index Stats Dimensionindex.describe_index_stats().dimensionVerifies vector dimension of connected index.
95Check Index Stats Fullnessindex.describe_index_stats().index_fullnessPulls pod memory capacity fullness ratio.
96Check Import Job Stateindex.describe_import(import_id='import-123').statusInspects status of bulk import job.
97Check Import Job Records Importedindex.describe_import(import_id='import-123').records_importedInspects count of imported records.
98Check Import Job Error Messageindex.describe_import(import_id='import-123').errorInspects error message if import failed.
99Check API Key Name List[key.name for key in pc.list_api_keys()]Extracts list of API key names.
100Verify Pinecone Client Readypc.list_indexes()Verifies active authentication with Pinecone cloud API.
101Initialize Pinecone Client Option 1from pinecone import Pinecone; pc = Pinecone(api_key="pc_secure_key")Authenticates Pinecone SDK session.
102Create Serverless Index Option 2pc.create_index(name="enterprise-index", dimension=1536, metric="cosine", spec=ServerlessSpec(cloud="aws", region="us-east-1"))Provisions managed serverless vector index.
103Delete Index Option 3pc.delete_index(name="enterprise-index")Deletes serverless vector index.
104List Indexes Option 4pc.list_indexes()Outputs all vector indexes.
105Describe Index Option 5pc.describe_index(name="enterprise-index")Inspects vector index dimensions and status.
106Connect to Index Host Option 6index = pc.Index("enterprise-index")Connects to specific vector index instance.
107Upsert Vectors Option 7index.upsert(vectors=[{"id": "vec1", "values": [0.1, ...], "metadata": {"category": "tech"}}])Writes vector embeddings and metadata payload.
108Query Vectors Option 8index.query(vector=[0.1, ...], top_k=5, include_metadata=True)Executes nearest-neighbor vector similarity search.
109Query with Metadata Filter Option 9index.query(vector=[0.1, ...], filter={"category": {"$eq": "tech"}}, top_k=5, include_metadata=True)Executes filtered vector search.
110Fetch Vectors by ID Option 10index.fetch(ids=["vec1"])Retrieves vectors by record ID.
111Delete Vectors by ID Option 11index.delete(ids=["vec1"])Removes vector records.
112Delete Vectors by Filter Option 12index.delete(filter={"category": {"$eq": "tech"}})Removes vectors matching metadata filter.
113Delete All Vectors (Clear) Option 13index.delete(delete_all=true)Clears all vectors from index.
114Check Index Statistics Option 14index.describe_index_stats()Calculates vector counts and namespace distribution.
115Create Collection (Backup) Option 15pc.create_collection(name="index-backup", source_index="enterprise-index")Takes serverless index snapshot collection.
116List Collections Option 16pc.list_collections()Outputs all index backup collections.
117Delete Collection Option 17pc.delete_collection(name="index-backup")Removes index backup collection.
118Create Pod-Based Index (GPU/CPU) Option 18pc.create_index(name="pod-index", dimension=1536, metric="cosine", spec=PodSpec(environment="us-west1-gcp", pod_type="p1.x1", pods=1))Provisions dedicated pod vector index.
119Configure Pod Index Scale Option 19pc.configure_index(name="pod-index", replicas=2, pod_type="p1.x2")Scales pod compute resources.
120Create Namespace Option 20index.upsert(vectors=[...], namespace="tenant-a")Provisions isolated tenant namespace.
121Query Specific Namespace Option 21index.query(vector=[0.1, ...], namespace="tenant-a", top_k=5)Executes vector search within tenant namespace.
122Update Vector Record Option 22index.update(id="vec1", values=[0.2, ...], set_metadata={"status": "updated"})Modifies existing vector values and metadata.
123Start Import Job (S3) Option 23index.start_import(uri="s3://my-bucket/vectors/")Provisions bulk vector import from cloud storage.
124List Import Jobs Option 24index.list_imports()Outputs bulk import job statuses.
125Cancel Import Job Option 25index.cancel_import(import_id="import-123")Stops active bulk vector import job.
126Check Import Job Status Option 26index.describe_import(import_id="import-123")Inspects vector import job progress.
127Create API Key Option 27pc.create_api_key(name="app-key")Provisions programmatic access token.
128List API Keys Option 28pc.list_api_keys()Outputs active API keys.
129Delete API Key Option 29pc.delete_api_key(api_key_id="key-123")Revokes API access token.
130Check Pinecone SDK Version Option 30import pinecone; print(pinecone.__version__)Outputs running Pinecone Python SDK version.
131Check Has Index Option 31pc.has_index(name="enterprise-index")Verifies whether vector index exists in account.
132Describe Collection Details Option 32pc.describe_collection(name="index-backup")Inspects size, record count, and source index of collection snapshot.
133Upsert Vectors with Namespace Option 33index.upsert(vectors=[...], namespace="tenant-b")Writes vectors into tenant-b isolated namespace.
134Query Vectors Include Values Option 34index.query(vector=[0.1, ...], top_k=5, include_values=True)Returns raw float vector array in query response.
135Query ID Search Option 35index.query(id="vec1", top_k=5, include_metadata=True)Executes vector search using vector ID as query vector.
136Query Filter In Operator Option 36index.query(vector=[0.1, ...], filter={"category": {"$in": ["tech", "finance"]}}, top_k=5)Filters search matching array of values.
137Query Filter Not In Operator Option 37index.query(vector=[0.1, ...], filter={"category": {"$nin": ["archived"]}}, top_k=5)Filters search excluding specified values.
138Query Filter Greater Than Operator Option 38index.query(vector=[0.1, ...], filter={"price": {"$gt": 100}}, top_k=5)Filters search where numeric property > 100.
139Query Filter Less Than Operator Option 39index.query(vector=[0.1, ...], filter={"price": {"$lte": 500}}, top_k=5)Filters search where numeric property <= 500.
140Query Filter And Operator Option 40index.query(vector=[0.1, ...], filter={"$and": [{"category": {"$eq": "tech"}}, {"price": {"$lt": 200}}]}, top_k=5)Filters search matching AND conditions.
141Query Filter Or Operator Option 41index.query(vector=[0.1, ...], filter={"$or": [{"category": {"$eq": "tech"}}, {"featured": {"$eq": true}}]}, top_k=5)Filters search matching OR conditions.
142Fetch Multiple Vectors Option 42index.fetch(ids=["vec1", "vec2", "vec3"], namespace="tenant-a")Batch fetches vectors by IDs from namespace.
143Delete Vectors in Namespace Option 43index.delete(delete_all=True, namespace="tenant-a")Purges all vectors from specific namespace.
144Update Vector Metadata Only Option 44index.update(id="vec1", set_metadata={"status": "verified"}, namespace="tenant-a")Updates metadata attributes without rewriting vector array.
145Update Vector Values Only Option 45index.update(id="vec1", values=[0.3, ...], namespace="tenant-a")Updates vector float array without changing metadata.
146List Vectors in Namespace (Paginated) Option 46for ids_batch in index.list(namespace="tenant-a"): print(ids_batch)Paginates through vector IDs in namespace.
147List Vectors with Prefix Filter Option 47index.list(prefix="user_", namespace="tenant-a")Lists vector IDs matching prefix string.
148Create Sparse-Dense Hybrid Index Option 48pc.create_index(name="hybrid-index", dimension=1536, metric="dotproduct", spec=ServerlessSpec(cloud="aws", region="us-east-1"))Provisions index supporting sparse-dense hybrid vectors.
149Upsert Sparse-Dense Vector Option 49index.upsert(vectors=[{"id": "vec1", "values": [0.1, ...], "sparse_values": {"indices": [1, 5], "values": [0.8, 0.4]}}])Writes combined sparse-dense vector embedding.
150Query Sparse-Dense Hybrid Search Option 50index.query(vector=[0.1, ...], sparse_vector=SparseValues(indices=[1, 5], values=[0.8, 0.4]), top_k=5)Executes hybrid vector similarity search.
151Configure Index Replicas Option 51pc.configure_index(name="pod-index", replicas=3)Scales read replica pods across availability zones.
152Configure Index Pod Type Option 52pc.configure_index(name="pod-index", pod_type="p2.x1")Scales up compute pod hardware type.
153Create Serverless Index GCP Option 53pc.create_index(name="gcp-index", dimension=1536, metric="cosine", spec=ServerlessSpec(cloud="gcp", region="us-central1"))Provisions serverless index in GCP region.
154Create Serverless Index Azure Option 54pc.create_index(name="azure-index", dimension=1536, metric="cosine", spec=ServerlessSpec(cloud="azure", region="eastus2"))Provisions serverless index in Azure region.
155Check Index Host URL Option 55index_info = pc.describe_index(name="enterprise-index"); print(index_info.host)Extracts gRPC/REST endpoint host URL.
156Connect Index via Direct Host URL Option 56index = pc.Index(host="https://enterprise-index-xyz.svc.us-east-1.pinecone.io")Connects to index using explicit endpoint URL.
157Check Index Deletion Protection Option 57pc.configure_index(name="enterprise-index", deletion_protection="enabled")Enforces deletion protection on index.
158Disable Index Deletion Protection Option 58pc.configure_index(name="enterprise-index", deletion_protection="disabled")Disables deletion protection on index.
159Create Index from Collection Backup Option 59pc.create_index(name="restored-index", dimension=1536, metric="cosine", spec=PodSpec(environment="us-west1-gcp", pod_type="p1.x1", source_collection="index-backup"))Restores vector index from collection snapshot.
160Check Collection Status Option 60collection_info = pc.describe_collection(name="index-backup"); print(collection_info.status)Verifies if collection snapshot status is Ready.
161Check Collection Size Bytes Option 61collection_info = pc.describe_collection(name="index-backup"); print(collection_info.size)Calculates physical storage bytes of collection snapshot.
162Check Index Vector Count Stats Option 62stats = index.describe_index_stats(); print(stats.total_vector_count)Calculates total vectors across all namespaces.
163Check Index Dimension Stats Option 63stats = index.describe_index_stats(); print(stats.dimension)Inspects vector dimension size.
164Check Index Namespaces List Option 64stats = index.describe_index_stats(); print(stats.namespaces)Lists all active namespaces and vector counts.
165Check Index Fullness Metric Option 65stats = index.describe_index_stats(); print(stats.index_fullness)Inspects pod capacity utilization percentage.
166Import Vectors Integration S3 Role Option 66index.start_import(uri="s3://my-bucket/vectors/", integration_id="int-123")Imports vectors using AWS IAM integration role.
167Check Import Errors Option 67job = index.describe_import(import_id="import-123"); print(job.error)Inspects bulk import error log.
168Check Import Records Processed Option 68job = index.describe_import(import_id="import-123"); print(job.records_imported)Inspects count of successfully imported vector records.
169Create Integration AWS IAM Option 69pc.create_integration(name="s3-int", type="s3", aws_role_arn="arn:aws:iam::123:role/PineconeRole")Provisions cloud storage integration.
170List Integrations Option 70pc.list_integrations()Lists registered cloud storage integrations.
171Describe Integration Option 71pc.describe_integration(name="s3-int")Inspects status of cloud storage integration.
172Delete Integration Option 72pc.delete_integration(name="s3-int")Deletes cloud storage integration.
173Check API Key Permissions Option 73pc.describe_api_key(api_key_id="key-123")Inspects scope and creation timestamp of API key.
174Check Account Metrics CloudWatch Option 74aws cloudwatch get-metric-data --metric-data-queries ...Pulls Pinecone vector search latency and QPS from CloudWatch.
175Check Index REST Search HTTP Option 75curl -X POST -H 'Api-Key: pc_secure_key' -H 'Content-Type: application/json' -d '{"vector": [0.1, ...], "topK": 5}' https://enterprise-index-xyz.svc.us-east-1.pinecone.io/queryExecutes REST API search request directly.
176Check Index REST Upsert HTTP Option 76curl -X POST -H 'Api-Key: pc_secure_key' -H 'Content-Type: application/json' -d '{"vectors": [{"id": "vec1", "values": [0.1, ...]}]}' https://enterprise-index-xyz.svc.us-east-1.pinecone.io/vectors/upsertExecutes REST API upsert request directly.
177Check Index REST Fetch HTTP Option 77curl -X GET -H 'Api-Key: pc_secure_key' 'https://enterprise-index-xyz.svc.us-east-1.pinecone.io/vectors/fetch?ids=vec1'Executes REST API fetch request directly.
178Check Index REST Delete HTTP Option 78curl -X POST -H 'Api-Key: pc_secure_key' -H 'Content-Type: application/json' -d '{"ids": ["vec1"]}' https://enterprise-index-xyz.svc.us-east-1.pinecone.io/vectors/deleteExecutes REST API delete request directly.
179Check Index REST Describe Stats HTTP Option 79curl -X POST -H 'Api-Key: pc_secure_key' https://enterprise-index-xyz.svc.us-east-1.pinecone.io/describe_index_statsExecutes REST API describe index stats request.
180Verify Pinecone gRPC Channel SSL Option 80index = pc.Index("enterprise-index", pool_threads=30)Provisions connection pool with 30 concurrent threads.
181Check Client User Agent String Option 81pc = Pinecone(api_key="pc_secure_key", source_tag="custom_app")Sets custom source tag in client user agent header.
182Close Pinecone Client Connection Option 82index.close()Closes active gRPC network channels to Pinecone index.
183Check Index Name List Option 83[idx.name for idx in pc.list_indexes()]Extracts list of index names.
184Check Collection Name List Option 84[col.name for idx in pc.list_collections()]Extracts list of collection snapshot names.
185Check Index Dimension Option 85pc.describe_index(name='enterprise-index').dimensionInspects vector dimension size.
186Check Index Metric Option 86pc.describe_index(name='enterprise-index').metricInspects vector distance metric (cosine/dotproduct/euclidean).
187Check Index Host Endpoint Option 87pc.describe_index(name='enterprise-index').hostExtracts index host endpoint URL.
188Check Index Status Ready Option 88pc.describe_index(name='enterprise-index').status.readyVerifies if index status is ready.
189Check Index Status State Option 89pc.describe_index(name='enterprise-index').status.stateInspects index operational state string.
190Check Index Spec Cloud Option 90pc.describe_index(name='enterprise-index').spec.serverless.cloudInspects serverless cloud provider (aws/gcp/azure).
191Check Index Spec Region Option 91pc.describe_index(name='enterprise-index').spec.serverless.regionInspects serverless cloud region.
192Check Index Stats Total Vector Count Option 92index.describe_index_stats().total_vector_countCalculates total vector count across index.
193Check Index Stats Namespaces Dict Option 93index.describe_index_stats().namespacesPulls dictionary of namespaces and vector counts.
194Check Index Stats Dimension Option 94index.describe_index_stats().dimensionVerifies vector dimension of connected index.
195Check Index Stats Fullness Option 95index.describe_index_stats().index_fullnessPulls pod memory capacity fullness ratio.
196Check Import Job State Option 96index.describe_import(import_id='import-123').statusInspects status of bulk import job.
197Check Import Job Records Imported Option 97index.describe_import(import_id='import-123').records_importedInspects count of imported records.
198Check Import Job Error Message Option 98index.describe_import(import_id='import-123').errorInspects error message if import failed.
199Check API Key Name List Option 99[key.name for key in pc.list_api_keys()]Extracts list of API key names.
200Verify Pinecone Client Ready Option 100pc.list_indexes()Verifies active authentication with Pinecone cloud API.