Is it possible to make asynchronous stored procedure calls to VoltDB using the Python client
Is it possible to make asynchronous calls to a stored procedure in VoltDB (an insert in a custom Java Stored procedure) using the Python client?
It looks like it isn't supported but is there a way to not wait for the response, or will I have to move to the Java client for async support?
python asynchronous voltdb
asked Nov 19 '18 at 18:59
StevieBStevieB
31
add a comment |
Is it possible to make asynchronous calls to a stored procedure in VoltDB (an insert in a custom Java Stored procedure) using the Python client?
It looks like it isn't supported but is there a way to not wait for the response, or will I have to move to the Java client for async support?
python asynchronous voltdb
asked Nov 19 '18 at 18:59
StevieBStevieB
31
add a comment |
Is it possible to make asynchronous calls to a stored procedure in VoltDB (an insert in a custom Java Stored procedure) using the Python client?
It looks like it isn't supported but is there a way to not wait for the response, or will I have to move to the Java client for async support?
python asynchronous voltdb
asked Nov 19 '18 at 18:59
StevieBStevieB
31
Is it possible to make asynchronous calls to a stored procedure in VoltDB (an insert in a custom Java Stored procedure) using the Python client?
It looks like it isn't supported but is there a way to not wait for the response, or will I have to move to the Java client for async support?
python asynchronous voltdb
python asynchronous voltdb
asked Nov 19 '18 at 18:59
StevieBStevieB
31
asked Nov 19 '18 at 18:59
StevieBStevieB
31
asked Nov 19 '18 at 18:59
StevieBStevieB
31
asked Nov 19 '18 at 18:59
StevieBStevieB
31
asked Nov 19 '18 at 18:59
StevieBStevieB
31
31
add a comment |
add a comment |
2 Answers
2
active
oldest
votes
The VoltDB python client does not support asynchronous calls. It may be possible to make calls from a multi-threaded python application, but we've never tested that, so I don't want to lead you into uncharted waters.
The java, C++, and Go clients support asynchronous calls.
If you're mainly trying to do fast inserts, you might leverage csvloader and you could probably execute csvloader from within a python application, but that's probably not what you're looking to do.
Disclosure: I work at VoltDB.
answered Nov 19 '18 at 20:34
BenjaminBallardBenjaminBallard
1,2671011
Thanks for the reply. I've tried from a multi threaded python application with 50-100 threads so far and it seems to work, but I haven't finished testing yet! We're looking to minimise latency on the inserts, would the csv loader increase latency if working in batches? Looks like will need to move to one of the other clients.
– StevieB
Nov 20 '18 at 11:28
For the same request, the latency should be the same regardless of which client is used. But asynchronous requests allow you to send many requests at once so that throughput is not a function of the latency. A multi-threaded client making synchronous requests can also achieve high throughput, but may require a lot of threads to be able to reach the same level as a single-threaded client sending asynchronous requests.
– BenjaminBallard
Nov 20 '18 at 14:07
CSVLoader would generally increase the latency, because it would be sending larger batch insert requests into the queues and these take longer to process than individual inserts. You can optionally call a procedure from CSVLoader using the -p parameter, which is usually slightly lower throughput, but can have less impact on latency.
– BenjaminBallard
Nov 20 '18 at 14:08
add a comment |
I hit this same issue building a Tornado process that is also a VoltDB client. Turned out it's quite easy to split VoltProcedure.call( ) into two funcs, the second to be invoked async by Tornado's ioloop when it detects a response on the socket. I also fixed a bug in writeDate( ). Take a look here: https://github.com/osullivj/voltdb-client-python
answered Dec 20 '18 at 20:17
osullivjosullivj
32715
add a comment |
Your Answer
StackExchange.ifUsing("editor", function () {
StackExchange.using("externalEditor", function () {
StackExchange.using("snippets", function () {
StackExchange.snippets.init();
});
});
}, "code-snippets");
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "1"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53381017%2fis-it-possible-to-make-asynchronous-stored-procedure-calls-to-voltdb-using-the-p%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
2 Answers
2
active
oldest
votes
2 Answers
2
active
oldest
votes
active
oldest
votes
active
oldest
votes
The VoltDB python client does not support asynchronous calls. It may be possible to make calls from a multi-threaded python application, but we've never tested that, so I don't want to lead you into uncharted waters.
The java, C++, and Go clients support asynchronous calls.
If you're mainly trying to do fast inserts, you might leverage csvloader and you could probably execute csvloader from within a python application, but that's probably not what you're looking to do.
Disclosure: I work at VoltDB.
answered Nov 19 '18 at 20:34
BenjaminBallardBenjaminBallard
1,2671011
Thanks for the reply. I've tried from a multi threaded python application with 50-100 threads so far and it seems to work, but I haven't finished testing yet! We're looking to minimise latency on the inserts, would the csv loader increase latency if working in batches? Looks like will need to move to one of the other clients.
– StevieB
Nov 20 '18 at 11:28
For the same request, the latency should be the same regardless of which client is used. But asynchronous requests allow you to send many requests at once so that throughput is not a function of the latency. A multi-threaded client making synchronous requests can also achieve high throughput, but may require a lot of threads to be able to reach the same level as a single-threaded client sending asynchronous requests.
– BenjaminBallard
Nov 20 '18 at 14:07
CSVLoader would generally increase the latency, because it would be sending larger batch insert requests into the queues and these take longer to process than individual inserts. You can optionally call a procedure from CSVLoader using the -p parameter, which is usually slightly lower throughput, but can have less impact on latency.
– BenjaminBallard
Nov 20 '18 at 14:08
add a comment |
The VoltDB python client does not support asynchronous calls. It may be possible to make calls from a multi-threaded python application, but we've never tested that, so I don't want to lead you into uncharted waters.
The java, C++, and Go clients support asynchronous calls.
If you're mainly trying to do fast inserts, you might leverage csvloader and you could probably execute csvloader from within a python application, but that's probably not what you're looking to do.
Disclosure: I work at VoltDB.
answered Nov 19 '18 at 20:34
BenjaminBallardBenjaminBallard
1,2671011
Thanks for the reply. I've tried from a multi threaded python application with 50-100 threads so far and it seems to work, but I haven't finished testing yet! We're looking to minimise latency on the inserts, would the csv loader increase latency if working in batches? Looks like will need to move to one of the other clients.
– StevieB
Nov 20 '18 at 11:28
For the same request, the latency should be the same regardless of which client is used. But asynchronous requests allow you to send many requests at once so that throughput is not a function of the latency. A multi-threaded client making synchronous requests can also achieve high throughput, but may require a lot of threads to be able to reach the same level as a single-threaded client sending asynchronous requests.
– BenjaminBallard
Nov 20 '18 at 14:07
CSVLoader would generally increase the latency, because it would be sending larger batch insert requests into the queues and these take longer to process than individual inserts. You can optionally call a procedure from CSVLoader using the -p parameter, which is usually slightly lower throughput, but can have less impact on latency.
– BenjaminBallard
Nov 20 '18 at 14:08
add a comment |
The VoltDB python client does not support asynchronous calls. It may be possible to make calls from a multi-threaded python application, but we've never tested that, so I don't want to lead you into uncharted waters.
The java, C++, and Go clients support asynchronous calls.
If you're mainly trying to do fast inserts, you might leverage csvloader and you could probably execute csvloader from within a python application, but that's probably not what you're looking to do.
Disclosure: I work at VoltDB.
answered Nov 19 '18 at 20:34
BenjaminBallardBenjaminBallard
1,2671011
The VoltDB python client does not support asynchronous calls. It may be possible to make calls from a multi-threaded python application, but we've never tested that, so I don't want to lead you into uncharted waters.
The java, C++, and Go clients support asynchronous calls.
If you're mainly trying to do fast inserts, you might leverage csvloader and you could probably execute csvloader from within a python application, but that's probably not what you're looking to do.
Disclosure: I work at VoltDB.
answered Nov 19 '18 at 20:34
BenjaminBallardBenjaminBallard
1,2671011
answered Nov 19 '18 at 20:34
BenjaminBallardBenjaminBallard
1,2671011
answered Nov 19 '18 at 20:34
BenjaminBallardBenjaminBallard
1,2671011
answered Nov 19 '18 at 20:34
BenjaminBallardBenjaminBallard
1,2671011
1,2671011
Thanks for the reply. I've tried from a multi threaded python application with 50-100 threads so far and it seems to work, but I haven't finished testing yet! We're looking to minimise latency on the inserts, would the csv loader increase latency if working in batches? Looks like will need to move to one of the other clients.
– StevieB
Nov 20 '18 at 11:28
For the same request, the latency should be the same regardless of which client is used. But asynchronous requests allow you to send many requests at once so that throughput is not a function of the latency. A multi-threaded client making synchronous requests can also achieve high throughput, but may require a lot of threads to be able to reach the same level as a single-threaded client sending asynchronous requests.
– BenjaminBallard
Nov 20 '18 at 14:07
CSVLoader would generally increase the latency, because it would be sending larger batch insert requests into the queues and these take longer to process than individual inserts. You can optionally call a procedure from CSVLoader using the -p parameter, which is usually slightly lower throughput, but can have less impact on latency.
– BenjaminBallard
Nov 20 '18 at 14:08
add a comment |
Thanks for the reply. I've tried from a multi threaded python application with 50-100 threads so far and it seems to work, but I haven't finished testing yet! We're looking to minimise latency on the inserts, would the csv loader increase latency if working in batches? Looks like will need to move to one of the other clients.
– StevieB
Nov 20 '18 at 11:28
For the same request, the latency should be the same regardless of which client is used. But asynchronous requests allow you to send many requests at once so that throughput is not a function of the latency. A multi-threaded client making synchronous requests can also achieve high throughput, but may require a lot of threads to be able to reach the same level as a single-threaded client sending asynchronous requests.
– BenjaminBallard
Nov 20 '18 at 14:07
CSVLoader would generally increase the latency, because it would be sending larger batch insert requests into the queues and these take longer to process than individual inserts. You can optionally call a procedure from CSVLoader using the -p parameter, which is usually slightly lower throughput, but can have less impact on latency.
– BenjaminBallard
Nov 20 '18 at 14:08
Thanks for the reply. I've tried from a multi threaded python application with 50-100 threads so far and it seems to work, but I haven't finished testing yet! We're looking to minimise latency on the inserts, would the csv loader increase latency if working in batches? Looks like will need to move to one of the other clients.
– StevieB
Nov 20 '18 at 11:28
Thanks for the reply. I've tried from a multi threaded python application with 50-100 threads so far and it seems to work, but I haven't finished testing yet! We're looking to minimise latency on the inserts, would the csv loader increase latency if working in batches? Looks like will need to move to one of the other clients.
– StevieB
Nov 20 '18 at 11:28
For the same request, the latency should be the same regardless of which client is used. But asynchronous requests allow you to send many requests at once so that throughput is not a function of the latency. A multi-threaded client making synchronous requests can also achieve high throughput, but may require a lot of threads to be able to reach the same level as a single-threaded client sending asynchronous requests.
– BenjaminBallard
Nov 20 '18 at 14:07
For the same request, the latency should be the same regardless of which client is used. But asynchronous requests allow you to send many requests at once so that throughput is not a function of the latency. A multi-threaded client making synchronous requests can also achieve high throughput, but may require a lot of threads to be able to reach the same level as a single-threaded client sending asynchronous requests.
– BenjaminBallard
Nov 20 '18 at 14:07
CSVLoader would generally increase the latency, because it would be sending larger batch insert requests into the queues and these take longer to process than individual inserts. You can optionally call a procedure from CSVLoader using the -p parameter, which is usually slightly lower throughput, but can have less impact on latency.
– BenjaminBallard
Nov 20 '18 at 14:08
CSVLoader would generally increase the latency, because it would be sending larger batch insert requests into the queues and these take longer to process than individual inserts. You can optionally call a procedure from CSVLoader using the -p parameter, which is usually slightly lower throughput, but can have less impact on latency.
– BenjaminBallard
Nov 20 '18 at 14:08
add a comment |
I hit this same issue building a Tornado process that is also a VoltDB client. Turned out it's quite easy to split VoltProcedure.call( ) into two funcs, the second to be invoked async by Tornado's ioloop when it detects a response on the socket. I also fixed a bug in writeDate( ). Take a look here: https://github.com/osullivj/voltdb-client-python
answered Dec 20 '18 at 20:17
osullivjosullivj
32715
add a comment |
I hit this same issue building a Tornado process that is also a VoltDB client. Turned out it's quite easy to split VoltProcedure.call( ) into two funcs, the second to be invoked async by Tornado's ioloop when it detects a response on the socket. I also fixed a bug in writeDate( ). Take a look here: https://github.com/osullivj/voltdb-client-python
answered Dec 20 '18 at 20:17
osullivjosullivj
32715
add a comment |
I hit this same issue building a Tornado process that is also a VoltDB client. Turned out it's quite easy to split VoltProcedure.call( ) into two funcs, the second to be invoked async by Tornado's ioloop when it detects a response on the socket. I also fixed a bug in writeDate( ). Take a look here: https://github.com/osullivj/voltdb-client-python
answered Dec 20 '18 at 20:17
osullivjosullivj
32715
I hit this same issue building a Tornado process that is also a VoltDB client. Turned out it's quite easy to split VoltProcedure.call( ) into two funcs, the second to be invoked async by Tornado's ioloop when it detects a response on the socket. I also fixed a bug in writeDate( ). Take a look here: https://github.com/osullivj/voltdb-client-python
answered Dec 20 '18 at 20:17
osullivjosullivj
32715
answered Dec 20 '18 at 20:17
osullivjosullivj
32715
answered Dec 20 '18 at 20:17
osullivjosullivj
32715
answered Dec 20 '18 at 20:17
osullivjosullivj
32715
32715
add a comment |
add a comment |
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53381017%2fis-it-possible-to-make-asynchronous-stored-procedure-calls-to-voltdb-using-the-p%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
