How to Deploy MinIO and Trino with Kubernetes

Trino (formerly Presto) is a SQL query engine - not a SQL database. Trino has eschewed the storage component of the SQL database to focus on only one thing - ultra-fast SQL querying. Trino is just a query engine and does not store data. Instead, Trino interacts with various databases or directly on object storage. Trino parses and analyzes the SQL query you pass in, creates and optimizes a query execution plan that includes the data sources, and then schedules worker nodes that are able to intelligently query the underlying databases they connect to. MinIO is frequently used to store data from AI/ML Workloads, Datalakes to lake houses whether it be Dremio, Hive, Hudi, StarRocks or any of the other dozen or so great AI/ML tools solutions. MinIO is more efficient when used as the primary storage layer, which decreases total cost of ownership for the data stored, plus you get the added benefits of writing data to MinIO that is immutable, versioned and protected by erasure coding. In addition, saving data to MinIO object storage makes it available to other cloud native machine learning and analytics applications. In this tutorial, we'll deploy a cohesive system that allows distributed SQL querying across large datasets stored in Minio, with Trino leveraging metadata from Hive Metastore and table schemas from Redis. Components Here are the different components and what they do in our setup process we’ll go through next. Minio: Minio can be used to store large datasets, like the ones typically analyzed by Trino. Hive Metastore: Hive Metastore is a service that stores metadata for Hive tables (like table schema). Trino can use Hive Metastore to determine the schema of tables when querying datasets. PostgreSQL for Hive Metastore: This is the database backend for the Hive Metastore. It's where the metadata is actually stored. Redis: In this setup, Redis for storing table schemas for Trino. Trino: Trino (formerly known as Presto) is a high-performance, distributed SQL query engine. It allows querying data across various data sources like SQL databases, NoSQL databases, and even object storage like Minio. Prerequisites Before starting, ensure you have the necessary tools installed for managing your Kubernetes cluster: kubectl: The primary command-line tool for managing Kubernetes clusters. You can use it to inspect, manipulate, and administer cluster resources. helm: A package manager for Kubernetes. Helm allows you to deploy, upgrade, and manage applications within your cluster using pre-defined charts. Repository Cloning To access the resources needed for deploying Trino on Kubernetes, clone the specific GitHub repository and navigate to the appropriate directory: git clone https://github.com/minio/blog-assets.git cd blog-assets/trino-on-kubernetes Kubernetes Namespace Creation Namespaces in Kubernetes provide isolated environments for applications. Create a new namespace for Trino to encapsulate its deployment: kubectl create namespace trino --dry-run=client -o yaml | kubectl apply -f - Redis Table Definition Secret Redis will store table schemas used by Trino. Secure these schemas with a Kubernetes Secret. The following command creates a generic secret, sourcing data from a JSON file: kubectl create secret generic redis-table-definition --from-file=redis/test.json -n trino || true Add Helm Repositories Helm repositories provide pre-packaged charts that simplify application deployment. Add the Bitnami and Trino repositories to your Helm configuration: helm repo add bitnami https://charts.bitnami.com/bitnami || true helm repo add trino https://trinodb.github.io/charts/ || true Deploy MinIO for Data Storage Initialize MinIO Prepare MinIO within the Trino namespace. kubectl minio init -n trino Create MinIO Tenant Set up a multi-tenant architecture for data storage. The example below creates a tenant named “tenant-1” with four servers, four storage volumes, and a capacity of 4 GiB: kubectl minio tenant create tenant-1 --servers 4 --volumes 4 --capacity 4Gi -n trino Set Up Hive Metastore Trino utilizes Hive Metastore to store table metadata. Deploy PostgreSQL to manage the metadata, then set up the Hive Metastore: Install PostgreSQL helm upgrade --install hive-metastore-postgresql bitnami/postgresql -n trino -f hive-metastore-postgresql/values.yaml Deploy Hive Metastore Use a preconfigured Helm chart to deploy Hive Metastore within the Trino namespace: helm upgrade --install my-hive-metastore -n trino -f hive-metastore/values.yaml ./charts/hive-metastore Deploying MinIO and Trino with Kubernetes Trino and MinIO create a powerful combination for distributed SQL querying across large datasets. Follow these steps to deploy and configure the system. Deploy Redis to Store Table Schemas Redis is a high-speed, in-memory data store used to hold Trino table schemas for enhanced query performance. Deploy it in the Trino namespace using a Helm chart: helm upgrade --install my-redis bitnami/redis -n trino -f redis/values.yaml Deploy Trino Deploy Trino as the distributed SQL query engine that will connect to MinIO and other data sources: helm upgrade --install my-trino trino/trino --version 0.7.0 --namespace trino -f trino/values.yaml Verify Deployment Confirm that all components are running correctly by listing the pods in the Trino namespace: kubectl get pods -n trino Security Review and Adjustments Review and adjust security settings as needed. To disable SSL certificate validation for S3 connections, update the additionalCatalogs section of the values.yaml file with the following property: hive.s3.ssl.enabled=false Testing Port Forward to MinIO Tenant Service Port forward to the MinIO service of the tenant, enabling local access: kubectl port-forward svc/minio -n trino 9443:443 Create Alias and Bucket for Trino 1. Create Alias: Establish an alias for the tenant using the credentials from the MinIO deployment: mc alias set my-minio https://localhost:9443/ minio_access_key minio_secret_key --insecure 2. Create Bucket: Create a new bucket that Trino will use mc mb my-minio/tiny --insecure Access Trino UI via Port Forward 1. Obtain Pod Name: Retrieve the name of the Trino coordinator pod: export POD_NAME=$(kubectl get pods --namespace trino -l "app=trino,release=my-trino,component=coordinator" -o jsonpath="{.items[0].metadata.name}") 2. Port Forward: Forward local port 8080 to the coordinator pod: kubectl port-forward $POD_NAME 8080:8080 3. Access UI: Use the Trino UI in your browser by visiting http://127.0.0.1:8080. Query Trino via CLI Access the Trino coordinator pod and start querying via the command line: kubectl exec -it deploy/my-trino-coordinator -n trino -- trino SHOW CATALOGS; SHOW SCHEMAS IN minio; Schema -------------------- default information_schema CREATE SCHEMA minio.tiny WITH (location = 's3a://tiny/'); CREATE TABLE minio.tiny.customer WITH ( format = 'ORC', external_location = 's3a://tiny/customer/' ) AS SELECT * FROM tpch.tiny.customer; SELECT * FROM minio.tiny.customer LIMIT 50; SHOW SCHEMAS IN minio; Schema -------------------- default information_schema tiny (3 rows) Confirm Data in MinIO Bucket After creating the bucket, confirm that the data is stored in MinIO by listing the contents with the mc command-line tool. Use the following command: mc ls my-minio/tiny --insecure It's as simple as that! Final Thoughts When troubleshooting configuration issues, especially those concerning security, thoroughly review the values.yaml files for each component to ensure proper settings. Trino stands out for its ability to optimize queries across various data layers, whether specialized databases or object storage. It aims to minimize data transfer by pushing down queries to retrieve only the essential data required. This enables Trino to join datasets from different sources, perform further processing, or return precise results efficiently. MinIO pairs exceptionally well with Trino due to its industry-leading scalability and performance. With the ability to handle significant workloads across AI/ML and analytics, MinIO effortlessly supports Trino queries and beyond. In recent benchmarks, MinIO achieved an impressive 325 GiB/s (349 GB/s) for GET operations and 165 GiB/s (177 GB/s) for PUT operations across just 32 nodes. This remarkable performance ensures that data stored in MinIO remains readily accessible, making MinIO a reliable and high-performing choice for Trino without becoming a bottleneck. If you have any questions on MinIO and Trino be sure to reach out to us on Slack! Trino (formerly Presto) is a SQL query engine - not a SQL database. Trino has eschewed the storage component of the SQL database to focus on only one thing - ultra-fast SQL querying. Trino is just a query engine and does not store data. Instead, Trino interacts with various databases or directly on object storage. Trino parses and analyzes the SQL query you pass in, creates and optimizes a query execution plan that includes the data sources, and then schedules worker nodes that are able to intelligently query the underlying databases they connect to. Trino Trino MinIO is frequently used to store data from AI/ML Workloads, Datalakes to lake houses whether it be Dremio, Hive, Hudi, StarRocks or any of the other dozen or so great AI/ML tools solutions. MinIO is more efficient when used as the primary storage layer, which decreases total cost of ownership for the data stored, plus you get the added benefits of writing data to MinIO that is immutable , versioned and protected by erasure coding . In addition, saving data to MinIO object storage makes it available to other cloud native machine learning and analytics applications. immutable immutable versioned versioned erasure coding erasure coding In this tutorial, we'll deploy a cohesive system that allows distributed SQL querying across large datasets stored in Minio, with Trino leveraging metadata from Hive Metastore and table schemas from Redis. Components Here are the different components and what they do in our setup process we’ll go through next. Minio: Minio can be used to store large datasets, like the ones typically analyzed by Trino. Hive Metastore: Hive Metastore is a service that stores metadata for Hive tables (like table schema). Trino can use Hive Metastore to determine the schema of tables when querying datasets. PostgreSQL for Hive Metastore: This is the database backend for the Hive Metastore. It's where the metadata is actually stored. Redis: In this setup, Redis for storing table schemas for Trino. Trino: Trino (formerly known as Presto) is a high-performance, distributed SQL query engine. It allows querying data across various data sources like SQL databases, NoSQL databases, and even object storage like Minio. Minio: Minio can be used to store large datasets, like the ones typically analyzed by Trino. Minio: Hive Metastore : Hive Metastore is a service that stores metadata for Hive tables (like table schema). Trino can use Hive Metastore to determine the schema of tables when querying datasets. Hive Metastore PostgreSQL for Hive Metastore: This is the database backend for the Hive Metastore. It's where the metadata is actually stored. PostgreSQL for Hive Metastore: Redis: In this setup, Redis for storing table schemas for Trino. Redis: Trino: Trino (formerly known as Presto) is a high-performance, distributed SQL query engine. It allows querying data across various data sources like SQL databases, NoSQL databases, and even object storage like Minio. Trino: Prerequisites Before starting, ensure you have the necessary tools installed for managing your Kubernetes cluster: kubectl: The primary command-line tool for managing Kubernetes clusters. You can use it to inspect, manipulate, and administer cluster resources. helm: A package manager for Kubernetes. Helm allows you to deploy, upgrade, and manage applications within your cluster using pre-defined charts. kubectl : The primary command-line tool for managing Kubernetes clusters. You can use it to inspect, manipulate, and administer cluster resources. kubectl helm : A package manager for Kubernetes. Helm allows you to deploy, upgrade, and manage applications within your cluster using pre-defined charts. helm Repository Cloning To access the resources needed for deploying Trino on Kubernetes, clone the specific GitHub repository and navigate to the appropriate directory: git clone https://github.com/minio/blog-assets.git cd blog-assets/trino-on-kubernetes git clone https://github.com/minio/blog-assets.git cd blog-assets/trino-on-kubernetes Kubernetes Namespace Creation Namespaces in Kubernetes provide isolated environments for applications. Create a new namespace for Trino to encapsulate its deployment: kubectl create namespace trino --dry-run=client -o yaml | kubectl apply -f - kubectl create namespace trino --dry-run=client -o yaml | kubectl apply -f - Redis Table Definition Secret Redis will store table schemas used by Trino. Secure these schemas with a Kubernetes Secret. The following command creates a generic secret, sourcing data from a JSON file: kubectl create secret generic redis-table-definition --from-file=redis/test.json -n trino || true kubectl create secret generic redis-table-definition --from-file=redis/test.json -n trino || true Add Helm Repositories Helm repositories provide pre-packaged charts that simplify application deployment. Add the Bitnami and Trino repositories to your Helm configuration: helm repo add bitnami https://charts.bitnami.com/bitnami || true helm repo add trino https://trinodb.github.io/charts/ || true helm repo add bitnami https://charts.bitnami.com/bitnami || true helm repo add trino https://trinodb.github.io/charts/ || true Deploy MinIO for Data Storage Initialize MinIO Prepare MinIO within the Trino namespace. kubectl minio init -n trino kubectl minio init -n trino Create MinIO Tenant Set up a multi-tenant architecture for data storage. The example below creates a tenant named “tenant-1” with four servers, four storage volumes, and a capacity of 4 GiB: kubectl minio tenant create tenant-1 --servers 4 --volumes 4 --capacity 4Gi -n trino kubectl minio tenant create tenant-1 --servers 4 --volumes 4 --capacity 4Gi -n trino Set Up Hive Metastore Trino utilizes Hive Metastore to store table metadata. Deploy PostgreSQL to manage the metadata, then set up the Hive Metastore: Install PostgreSQL helm upgrade --install hive-metastore-postgresql bitnami/postgresql -n trino -f hive-metastore-postgresql/values.yaml helm upgrade --install hive-metastore-postgresql bitnami/postgresql -n trino -f hive-metastore-postgresql/values.yaml Deploy Hive Metastore Use a preconfigured Helm chart to deploy Hive Metastore within the Trino namespace: helm upgrade --install my-hive-metastore -n trino -f hive-metastore/values.yaml ./charts/hive-metastore helm upgrade --install my-hive-metastore -n trino -f hive-metastore/values.yaml ./charts/hive-metastore Deploying MinIO and Trino with Kubernetes Trino and MinIO create a powerful combination for distributed SQL querying across large datasets. Follow these steps to deploy and configure the system. Deploy Redis to Store Table Schemas Redis is a high-speed, in-memory data store used to hold Trino table schemas for enhanced query performance. Deploy it in the Trino namespace using a Helm chart: helm upgrade --install my-redis bitnami/redis -n trino -f redis/values.yaml helm upgrade --install my-redis bitnami/redis -n trino -f redis/values.yaml Deploy Trino Deploy Trino as the distributed SQL query engine that will connect to MinIO and other data sources: helm upgrade --install my-trino trino/trino --version 0.7.0 --namespace trino -f trino/values.yaml helm upgrade --install my-trino trino/trino --version 0.7.0 --namespace trino -f trino/values.yaml Verify Deployment Confirm that all components are running correctly by listing the pods in the Trino namespace: kubectl get pods -n trino kubectl get pods -n trino Security Review and Adjustments Review and adjust security settings as needed. To disable SSL certificate validation for S3 connections, update the additionalCatalogs section of the values.yaml file with the following property: hive.s3.ssl.enabled=false hive.s3.ssl.enabled=false Testing Port Forward to MinIO Tenant Service Port forward to the MinIO service of the tenant, enabling local access: kubectl port-forward svc/minio -n trino 9443:443 kubectl port-forward svc/minio -n trino 9443:443 Create Alias and Bucket for Trino 1. Create Alias: Establish an alias for the tenant using the credentials from the MinIO deployment: 1. Create Alias: mc alias set my-minio https://localhost:9443/ minio_access_key minio_secret_key --insecure mc alias set my-minio https://localhost:9443/ minio_access_key minio_secret_key --insecure 2. Create Bucket: Create a new bucket that Trino will use 2. Create Bucket: mc mb my-minio/tiny --insecure mc mb my-minio/tiny --insecure Access Trino UI via Port Forward 1. Obtain Pod Name: Retrieve the name of the Trino coordinator pod: 1. Obtain Pod Name: export POD_NAME=$(kubectl get pods --namespace trino -l "app=trino,release=my-trino,component=coordinator" -o jsonpath="{.items[0].metadata.name}") export POD_NAME=$(kubectl get pods --namespace trino -l "app=trino,release=my-trino,component=coordinator" -o jsonpath="{.items[0].metadata.name}") 2. Port Forward: Forward local port 8080 to the coordinator pod: 2. Port Forward: kubectl port-forward $POD_NAME 8080:8080 kubectl port-forward $POD_NAME 8080:8080 3. Access UI: Use the Trino UI in your browser by visiting http://127.0.0.1:8080. 3. Access UI: Query Trino via CLI Access the Trino coordinator pod and start querying via the command line: kubectl exec -it deploy/my-trino-coordinator -n trino -- trino kubectl exec -it deploy/my-trino-coordinator -n trino -- trino SHOW CATALOGS; SHOW CATALOGS; SHOW SCHEMAS IN minio; SHOW SCHEMAS IN minio; Schema -------------------- default information_schema Schema -------------------- default information_schema CREATE SCHEMA minio.tiny WITH (location = 's3a://tiny/'); CREATE SCHEMA minio.tiny WITH (location = 's3a://tiny/'); CREATE TABLE minio.tiny.customer WITH ( format = 'ORC', external_location = 's3a://tiny/customer/' ) AS SELECT * FROM tpch.tiny.customer; CREATE TABLE minio.tiny.customer WITH ( format = 'ORC', external_location = 's3a://tiny/customer/' ) AS SELECT * FROM tpch.tiny.customer; SELECT * FROM minio.tiny.customer LIMIT 50; SELECT * FROM minio.tiny.customer LIMIT 50; SHOW SCHEMAS IN minio; SHOW SCHEMAS IN minio; Schema -------------------- default information_schema tiny (3 rows) Schema -------------------- default information_schema tiny (3 rows) Confirm Data in MinIO Bucket After creating the bucket, confirm that the data is stored in MinIO by listing the contents with the mc command-line tool. Use the following command: mc ls my-minio/tiny --insecure mc ls my-minio/tiny --insecure It's as simple as that! Final Thoughts When troubleshooting configuration issues, especially those concerning security, thoroughly review the values.yaml files for each component to ensure proper settings. Trino stands out for its ability to optimize queries across various data layers, whether specialized databases or object storage. It aims to minimize data transfer by pushing down queries to retrieve only the essential data required. This enables Trino to join datasets from different sources, perform further processing, or return precise results efficiently. MinIO pairs exceptionally well with Trino due to its industry-leading scalability and performance. With the ability to handle significant workloads across AI/ML and analytics, MinIO effortlessly supports Trino queries and beyond. In recent benchmarks, MinIO achieved an impressive 325 GiB/s (349 GB/s) for GET operations and 165 GiB/s (177 GB/s) for PUT operations across just 32 nodes. This remarkable performance ensures that data stored in MinIO remains readily accessible, making MinIO a reliable and high-performing choice for Trino without becoming a bottleneck. If you have any questions on MinIO and Trino be sure to reach out to us on Slack ! Slack Slack