Accessing Presto through Gateway

Common Cluster

Configure HAProxy

#---------------------------------------------------------------------# Global settings#---------------------------------------------------------------------global......## Configure the proxy to map the port 9090 of Gateway## to the port 9090 of emr-header-1.cluster-xxxxlisten prestojdbc: 9090mode tcpoption tcplogbalance sourceserver presto-coodinator-1 emr-header-1.cluster-xxxx:9090
$> service haproxy restart

Configure Security Groups

  • For an example of the command line using Presto, see here: Presto CLI
  • For an example of JDBC accessing Presto, see here: Presto JDBC

High Security Cluster

HTTPs Authentication

[root@emr-header-1 presto-conf]# keytool -genkey -dname "CN=emr-header-1.cluster-xxx,OU=Alibaba,O=Alibaba,L=HZ, ST=zhejiang, C=CN" -alias server -keyalg RSA -keystore keystore -keypass 81ba14ce6084 -storepass 81ba14ce6084 -validity 36500Warning:The JKS keystore uses a proprietary format. It is recommended to migrate to PKCS12 which is an industry standard format using "keytool -importkeystore -srckeystore keystore -destkeystore keystore -deststoretype pkcs12".
[root@emr-header-1 presto-conf]# keytool -export -alias server -file server.cer -keystore keystore -storepass 81ba14ce6084The certificate stored in the file <server.cer>Warning:The JKS keystore uses a proprietary format. It is recommended to migrate to PKCS12 which is an industry standard format using "keytool -importkeystore -srckeystore keystore -destkeystore keystore -deststoretype pkcs12".
[root@emr-header-1 presto-conf]# keytool -genkey -dname "CN=myhost,OU=Alibaba,O=Alibaba,L=HZ, ST=zhejiang, C=CN" -alias client -keyalg RSA -keystore client.keystore -keypass 123456 -storepass 123456 -validity 36500Warning:The JKS keystore uses a proprietary format. It is recommended to migrate to PKCS12, which is an industry standard format, using "keytool -importkeystore -srckeystore client.keystore -destkeystore client.keystore -deststoretype pkcs12".
[root@emr-header-1 presto-conf]# keytool -import -alias server -keystore client.keystore -file server.cer -storepass 123456Owner: CN=emr-header-2.cluster-xxx, OU=Alibaba, O=Alibaba, L=HZ, ST=zhejiang, C=CNPublisher: CN=emr-header-2.cluster-xxx, OU=Alibaba, O=Alibaba, L=HZ, ST=zhejiang, C=CNSerial number:4247108Validity period: Thu Mar 01 09:11:31 CST 2018 to Sat Feb 05 09:11:31 CST 2118Certificate fingerprint:MD5:  75:2A:AA:40:01:5B:3F:86:8F:9A:DB:B1:85:BD:44:8ASHA1: C7:25:B9:AD:5F:FE:FC:05:8E:A0:24:4A:1C:AA:6A:8D:6C:39:28:16SHA256: DB:86:69:65:73:D5:C6:E2:98:7C:4A:3B:31:EF:70:80:F0:3C:3B:0C:14:94:37:9F:9C:22:47:EA:7E:1E:DE:8CName of the signature algorithm: SHA256withRSASubject public key algorithm: 2048bit RSA keyVersion: 3Extension:#1: ObjectId: 2.5.29.14 Criticality=falseSubjectKeyIdentifier [KeyIdentifier [0000: 45 1D A9 C7 D5 4E BB CF  BD CE B4 5E E2 16 FB 2F  E.... N..... ^... /0010: E9 5D 4A B6  .] J.]]Do you trust this certificate? [No]: YesThe certificate has been added to the keystoreWarning:The JKS keystore uses a proprietary format. It is recommended to migrate to PKCS12, which is an industry standard format, using "keytool -importkeystore -srckeystore client.keystore -destkeystore client.keystore -deststoretype pkcs12".
$> scp root@xxx.xxx.xxx.xxx:/etc/ecm/presto-conf/client.keystore . /

Kerberos Authentication

[root@emr-header-1 presto-conf]# sh /usr/lib/has-current/bin/hadmin-local.sh /etc/ecm/has-conf -k /etc/ecm/has-conf/admin.keytab[INFO] conf_dir=/etc/ecm/has-confDebug is  true storeKey true useTicketCache false useKeyTab true doNotPrompt true ticketCache is null is Initiator true KeyTab is /etc/ecm/has-conf/admin.keytab refreshKrb5Config is true principal is kadmin/EMR.xxx.COM@EMR.xxx.COM tryFirstPass is false useFirstPass is false storePass is false clearPass is falseRefreshing Kerberos configurationprincipal is kadmin/EMR.xxx.COM@EMR.xxx.COMWill use keytabCommit SucceededLogin successful for user: kadmin/EMR.xxx.COM@EMR.xxx.COMenter "cmd" to see legal commands.HadminLocalTool.local: addprinc -pw 123456 clientuserSuccess to add principal: clientuserHadminLocalTool.local: ktadd -k /root/clientuser.keytab clientuserPrincipal export to keytab file: /root/clientuser.keytab successful .HadminLocalTool.local: exit
$> scp root@xxx.xxx.xxx.xxx:/root/clientuser.keytab . /$> scp root@xxx.xxx.xxx.xxx:/etc/krb5.conf . /
[libdefaults]kdc_realm = EMR.xxx.COMdefault_realm = EMR.xxx.COM# Change to 1, so that the client can use TCP protocol to communicate with KDC (because HAProxy does not support UDP protocol)udp_preference_limit = 1kdc_tcp_port = 88kdc_udp_port = 88dns_lookup_kdc = false[realms]EMR.xxx.COM = {# Set to the Internet IP of the Gatewaykdc = xxx.xxx.xxx.xxx:88}
#  gateway ipxxx.xxx.xxx.xxx emr-header-1.cluster-xxx

Configure Gateway HAProxy

#---------------------------------------------------------------------# Global settings#---------------------------------------------------------------------global......listen prestojdbc :7778mode tcpoption tcplogbalance sourceserver presto-coodinator-1 emr-header-1.cluster-xxx:7778listen kdc :88mode tcpoption tcplogbalance sourceserver emr-kdc emr-header-1:88
$> service haproxy restart

Configure Security Group Rules

Example of Using JDBC to Access Presto

try {Class.forName("com.facebook.presto.jdbc.PrestoDriver");} catch(ClassNotFoundException e) {LOG.error("Failed to load presto jdbc driver.", e);System.exit(-1);}Connection connection = null;Statement statement = null;try {String url = "jdbc:presto://emr-header-1.cluster-59824:7778/hive/default";Properties properties = new Properties();properties.setProperty("user", "hadoop");// Https related configurationproperties.setProperty("SSL", "true");properties.setProperty("SSLTrustStorePath", "resources/59824/client.keystore");properties.setProperty("SSLTrustStorePassword", "123456");// Kerberos related configurationproperties.setProperty("KerberosRemoteServiceName", "presto");properties.setProperty("KerberosPrincipal", "clientuser@EMR. 59824. COM");properties.setProperty("KerberosConfigPath", "resources/59824/krb5.conf");properties.setProperty("KerberosKeytabPath", "resources/59824/clientuser.keytab");// Create a Connection objectconnection = DriverManager.getConnection(url, properties);// Create a Statement objectstatement = connection.createStatement();// Execute the queryResultSet rs = statement.executeQuery("select * from table1");// Obtain the resultint columnNum = rs.getMetaData().getColumnCount();int rowIndex = 0;while (rs.next()) {rowIndex++;for(int i = 1; i <= columnNum; i++) {System.out.println("Row " + rowIndex + ", Column " + i + ": " + rs.getString(i));}}} catch(SQLException e) {LOG.error("Exception thrown.", e);} finally {// Destroy the Statement objectif (statement ! = null) {try {statement.close();} catch(Throwable t) {// No-ops}}// Close the Connectionif (connection ! = null) {try {connection.close();} catch(Throwable t) {// No-ops}}}

Summary

--

--

--

Follow me to keep abreast with the latest technology news, industry insights, and developer trends. Alibaba Cloud website:https://www.alibabacloud.com

Love podcasts or audiobooks? Learn on the go with our new app.

Recommended from Medium

What is Web 3.0 ?

Automation with the Integration of Jenkins and Docker

Editing Images Using Pixel Manipulation

Hello World! in ASM x86_64

How GCP is Different from AWS ?

Weeknotes 2020 week 8

What happens when you type `ls -l *.c` in the shell?

AWS Knowledge Series: Batch Processing using AWS Lambda

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Alibaba Cloud

Alibaba Cloud

Follow me to keep abreast with the latest technology news, industry insights, and developer trends. Alibaba Cloud website:https://www.alibabacloud.com

More from Medium

Stream avro data from kafka over ssl to Apache pinot

Automatically restarting failed Kafka Connectors and Connect Tasks

Apache Kafka

Use your own connector with Twitter and Aiven for Apache Kafka®