How can I get external table jdbc url in SnappyData
Previously I created an external table in SnappyData like this:
create external table EXT_DIM_CITY
using jdbc options(url 'jdbc:mysql://***:5002/***?user=***&password=***',
driver 'com.mysql.jdbc.Driver',
dbtable 'dim_city');
but now I forget the mysql jdbc url that EXT_DIM_CITY referred to. How can I get the jdbc url from SnappyData?
apache-spark jdbc external snappydata
edited Nov 20 '18 at 9:13
Brian Tompsett - 汤莱恩
4,2331338101
asked Nov 20 '18 at 8:09
Jiang YanJiang Yan
163
add a comment |
Previously I created an external table in SnappyData like this:
create external table EXT_DIM_CITY
using jdbc options(url 'jdbc:mysql://***:5002/***?user=***&password=***',
driver 'com.mysql.jdbc.Driver',
dbtable 'dim_city');
but now I forget the mysql jdbc url that EXT_DIM_CITY referred to. How can I get the jdbc url from SnappyData?
apache-spark jdbc external snappydata
edited Nov 20 '18 at 9:13
Brian Tompsett - 汤莱恩
4,2331338101
asked Nov 20 '18 at 8:09
Jiang YanJiang Yan
163
add a comment |
Previously I created an external table in SnappyData like this:
create external table EXT_DIM_CITY
using jdbc options(url 'jdbc:mysql://***:5002/***?user=***&password=***',
driver 'com.mysql.jdbc.Driver',
dbtable 'dim_city');
but now I forget the mysql jdbc url that EXT_DIM_CITY referred to. How can I get the jdbc url from SnappyData?
apache-spark jdbc external snappydata
edited Nov 20 '18 at 9:13
Brian Tompsett - 汤莱恩
4,2331338101
asked Nov 20 '18 at 8:09
Jiang YanJiang Yan
163
Previously I created an external table in SnappyData like this:
create external table EXT_DIM_CITY
using jdbc options(url 'jdbc:mysql://***:5002/***?user=***&password=***',
driver 'com.mysql.jdbc.Driver',
dbtable 'dim_city');
but now I forget the mysql jdbc url that EXT_DIM_CITY referred to. How can I get the jdbc url from SnappyData?
apache-spark jdbc external snappydata
apache-spark jdbc external snappydata
edited Nov 20 '18 at 9:13
Brian Tompsett - 汤莱恩
4,2331338101
asked Nov 20 '18 at 8:09
Jiang YanJiang Yan
163
edited Nov 20 '18 at 9:13
Brian Tompsett - 汤莱恩
4,2331338101
asked Nov 20 '18 at 8:09
Jiang YanJiang Yan
163
edited Nov 20 '18 at 9:13
Brian Tompsett - 汤莱恩
4,2331338101
edited Nov 20 '18 at 9:13
Brian Tompsett - 汤莱恩
4,2331338101
edited Nov 20 '18 at 9:13
Brian Tompsett - 汤莱恩
4,2331338101
4,2331338101
asked Nov 20 '18 at 8:09
Jiang YanJiang Yan
163
asked Nov 20 '18 at 8:09
Jiang YanJiang Yan
163
asked Nov 20 '18 at 8:09
Jiang YanJiang Yan
163
163
add a comment |
add a comment |
1 Answer
1
active
oldest
votes
With the latest SnappyData release 1.0.2.1, all table properties can be seen with extended describe:
describe extended EXT_DIM_CITY
The properties will be visible under the "# Detailed Table Information" line that has the "Properties: " tag. Note that when running from snappy shell, you will need to increase the maximum display width to see the full value of the string column (maximumdisplaywidth 2000).
However, in this case the url property value is deliberately masked out as "###" because it contains the embedded password. If you had the "user" and "password" options specified separately, then only the "password" property would have been masked and url would be visible.
So in this case you can instead write a job to force display the value directly using catalog API like below (Scala code):
package test
import java.io.PrintWriter
import com.typesafe.config.Config
import org.apache.spark.sql.catalyst.TableIdentifier
import org.apache.spark.sql._
object CatalogReadJob extends SnappySQLJob {
override def runSnappyJob(session: SnappySession, jobConfig: Config): Any = {
val catalog = session.sessionCatalog
val metadata = catalog.getTableMetadata(new TableIdentifier("EXT_DIM_CITY"))
// dump metadata properties to a file
new PrintWriter("/tmp/EXT_DIM_CITY-metadata.txt") {
write(metadata.toString() + "nFull URL = " + metadata.storage.properties("url"))
close()
}
}
override def isValidJob(ss: SnappySession, conf: Config): SnappyJobValidation = SnappyJobValid()
}
Use Some("schema") in TableIdentifier constructor above if the table is in a schema other than the default "APP". Compile the code using build tools like gradle/maven etc or directly using scalac: scalac -classpath '/path/to/product/jars/*' CatalogReadJob.scala
Create a jar, say test.jar, then submit: snappy-job.sh submit --lead :8090 --app-name CatalogRead --class test.CatalogReadJob --app-jar test.jar
The URL and all other table properties should be dumped in /tmp/EXT_DIM_CITY-metadata.txt
answered Dec 11 '18 at 17:25
SumedhSumedh
35317
add a comment |
Your Answer
StackExchange.ifUsing("editor", function () {
StackExchange.using("externalEditor", function () {
StackExchange.using("snippets", function () {
StackExchange.snippets.init();
});
});
}, "code-snippets");
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "1"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53388666%2fhow-can-i-get-external-table-jdbc-url-in-snappydata%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
With the latest SnappyData release 1.0.2.1, all table properties can be seen with extended describe:
describe extended EXT_DIM_CITY
The properties will be visible under the "# Detailed Table Information" line that has the "Properties: " tag. Note that when running from snappy shell, you will need to increase the maximum display width to see the full value of the string column (maximumdisplaywidth 2000).
However, in this case the url property value is deliberately masked out as "###" because it contains the embedded password. If you had the "user" and "password" options specified separately, then only the "password" property would have been masked and url would be visible.
So in this case you can instead write a job to force display the value directly using catalog API like below (Scala code):
package test
import java.io.PrintWriter
import com.typesafe.config.Config
import org.apache.spark.sql.catalyst.TableIdentifier
import org.apache.spark.sql._
object CatalogReadJob extends SnappySQLJob {
override def runSnappyJob(session: SnappySession, jobConfig: Config): Any = {
val catalog = session.sessionCatalog
val metadata = catalog.getTableMetadata(new TableIdentifier("EXT_DIM_CITY"))
// dump metadata properties to a file
new PrintWriter("/tmp/EXT_DIM_CITY-metadata.txt") {
write(metadata.toString() + "nFull URL = " + metadata.storage.properties("url"))
close()
}
}
override def isValidJob(ss: SnappySession, conf: Config): SnappyJobValidation = SnappyJobValid()
}
Use Some("schema") in TableIdentifier constructor above if the table is in a schema other than the default "APP". Compile the code using build tools like gradle/maven etc or directly using scalac: scalac -classpath '/path/to/product/jars/*' CatalogReadJob.scala
Create a jar, say test.jar, then submit: snappy-job.sh submit --lead :8090 --app-name CatalogRead --class test.CatalogReadJob --app-jar test.jar
The URL and all other table properties should be dumped in /tmp/EXT_DIM_CITY-metadata.txt
answered Dec 11 '18 at 17:25
SumedhSumedh
35317
add a comment |
With the latest SnappyData release 1.0.2.1, all table properties can be seen with extended describe:
describe extended EXT_DIM_CITY
The properties will be visible under the "# Detailed Table Information" line that has the "Properties: " tag. Note that when running from snappy shell, you will need to increase the maximum display width to see the full value of the string column (maximumdisplaywidth 2000).
However, in this case the url property value is deliberately masked out as "###" because it contains the embedded password. If you had the "user" and "password" options specified separately, then only the "password" property would have been masked and url would be visible.
So in this case you can instead write a job to force display the value directly using catalog API like below (Scala code):
package test
import java.io.PrintWriter
import com.typesafe.config.Config
import org.apache.spark.sql.catalyst.TableIdentifier
import org.apache.spark.sql._
object CatalogReadJob extends SnappySQLJob {
override def runSnappyJob(session: SnappySession, jobConfig: Config): Any = {
val catalog = session.sessionCatalog
val metadata = catalog.getTableMetadata(new TableIdentifier("EXT_DIM_CITY"))
// dump metadata properties to a file
new PrintWriter("/tmp/EXT_DIM_CITY-metadata.txt") {
write(metadata.toString() + "nFull URL = " + metadata.storage.properties("url"))
close()
}
}
override def isValidJob(ss: SnappySession, conf: Config): SnappyJobValidation = SnappyJobValid()
}
Use Some("schema") in TableIdentifier constructor above if the table is in a schema other than the default "APP". Compile the code using build tools like gradle/maven etc or directly using scalac: scalac -classpath '/path/to/product/jars/*' CatalogReadJob.scala
Create a jar, say test.jar, then submit: snappy-job.sh submit --lead :8090 --app-name CatalogRead --class test.CatalogReadJob --app-jar test.jar
The URL and all other table properties should be dumped in /tmp/EXT_DIM_CITY-metadata.txt
answered Dec 11 '18 at 17:25
SumedhSumedh
35317
add a comment |
With the latest SnappyData release 1.0.2.1, all table properties can be seen with extended describe:
describe extended EXT_DIM_CITY
The properties will be visible under the "# Detailed Table Information" line that has the "Properties: " tag. Note that when running from snappy shell, you will need to increase the maximum display width to see the full value of the string column (maximumdisplaywidth 2000).
However, in this case the url property value is deliberately masked out as "###" because it contains the embedded password. If you had the "user" and "password" options specified separately, then only the "password" property would have been masked and url would be visible.
So in this case you can instead write a job to force display the value directly using catalog API like below (Scala code):
package test
import java.io.PrintWriter
import com.typesafe.config.Config
import org.apache.spark.sql.catalyst.TableIdentifier
import org.apache.spark.sql._
object CatalogReadJob extends SnappySQLJob {
override def runSnappyJob(session: SnappySession, jobConfig: Config): Any = {
val catalog = session.sessionCatalog
val metadata = catalog.getTableMetadata(new TableIdentifier("EXT_DIM_CITY"))
// dump metadata properties to a file
new PrintWriter("/tmp/EXT_DIM_CITY-metadata.txt") {
write(metadata.toString() + "nFull URL = " + metadata.storage.properties("url"))
close()
}
}
override def isValidJob(ss: SnappySession, conf: Config): SnappyJobValidation = SnappyJobValid()
}
Use Some("schema") in TableIdentifier constructor above if the table is in a schema other than the default "APP". Compile the code using build tools like gradle/maven etc or directly using scalac: scalac -classpath '/path/to/product/jars/*' CatalogReadJob.scala
Create a jar, say test.jar, then submit: snappy-job.sh submit --lead :8090 --app-name CatalogRead --class test.CatalogReadJob --app-jar test.jar
The URL and all other table properties should be dumped in /tmp/EXT_DIM_CITY-metadata.txt
answered Dec 11 '18 at 17:25
SumedhSumedh
35317
With the latest SnappyData release 1.0.2.1, all table properties can be seen with extended describe:
describe extended EXT_DIM_CITY
The properties will be visible under the "# Detailed Table Information" line that has the "Properties: " tag. Note that when running from snappy shell, you will need to increase the maximum display width to see the full value of the string column (maximumdisplaywidth 2000).
However, in this case the url property value is deliberately masked out as "###" because it contains the embedded password. If you had the "user" and "password" options specified separately, then only the "password" property would have been masked and url would be visible.
So in this case you can instead write a job to force display the value directly using catalog API like below (Scala code):
package test
import java.io.PrintWriter
import com.typesafe.config.Config
import org.apache.spark.sql.catalyst.TableIdentifier
import org.apache.spark.sql._
object CatalogReadJob extends SnappySQLJob {
override def runSnappyJob(session: SnappySession, jobConfig: Config): Any = {
val catalog = session.sessionCatalog
val metadata = catalog.getTableMetadata(new TableIdentifier("EXT_DIM_CITY"))
// dump metadata properties to a file
new PrintWriter("/tmp/EXT_DIM_CITY-metadata.txt") {
write(metadata.toString() + "nFull URL = " + metadata.storage.properties("url"))
close()
}
}
override def isValidJob(ss: SnappySession, conf: Config): SnappyJobValidation = SnappyJobValid()
}
Use Some("schema") in TableIdentifier constructor above if the table is in a schema other than the default "APP". Compile the code using build tools like gradle/maven etc or directly using scalac: scalac -classpath '/path/to/product/jars/*' CatalogReadJob.scala
Create a jar, say test.jar, then submit: snappy-job.sh submit --lead :8090 --app-name CatalogRead --class test.CatalogReadJob --app-jar test.jar
The URL and all other table properties should be dumped in /tmp/EXT_DIM_CITY-metadata.txt
answered Dec 11 '18 at 17:25
SumedhSumedh
35317
answered Dec 11 '18 at 17:25
SumedhSumedh
35317
answered Dec 11 '18 at 17:25
SumedhSumedh
35317
answered Dec 11 '18 at 17:25
SumedhSumedh
35317
35317
add a comment |
add a comment |
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53388666%2fhow-can-i-get-external-table-jdbc-url-in-snappydata%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
