How to terminate an event loop
I have the following code in a django view to create a background task:
loop = asyncio.new_event_loop()
asyncio.set_event_loop(loop)
loop.run_in_executor(None, update_contacts, {
'email': email,
'access_token': g.tokens['access_token']
})
Is there anything I need to do at the end to 'kill' the loop? What would be the proper way to close it, etc?
python python-3.x python-asyncio
add a comment |
I have the following code in a django view to create a background task:
loop = asyncio.new_event_loop()
asyncio.set_event_loop(loop)
loop.run_in_executor(None, update_contacts, {
'email': email,
'access_token': g.tokens['access_token']
})
Is there anything I need to do at the end to 'kill' the loop? What would be the proper way to close it, etc?
python python-3.x python-asyncio
1
What you're designing there isfutures
. I believe you are usingThreadPoolExecutor
. You need to design it to be stoppable (eg. throw some flag/condition which can be interpreted for done) in order to finish and eventually let the daemon get shut down. Else, the event loop will live forever.
– Pruthvi Kumar
Nov 19 '18 at 6:15
1
Why are people lately hell-bent on usingasyncio
just to launch threads without anyasync
code? It is not whatasyncio
is there for, and using a plain Executor is much simpler.
– MisterMiyagi
Nov 19 '18 at 14:44
@MisterMiyagi perhaps you could show an example of how you'd suggest using it? I'm new to this all.
– David L
Nov 19 '18 at 19:46
add a comment |
I have the following code in a django view to create a background task:
loop = asyncio.new_event_loop()
asyncio.set_event_loop(loop)
loop.run_in_executor(None, update_contacts, {
'email': email,
'access_token': g.tokens['access_token']
})
Is there anything I need to do at the end to 'kill' the loop? What would be the proper way to close it, etc?
python python-3.x python-asyncio
I have the following code in a django view to create a background task:
loop = asyncio.new_event_loop()
asyncio.set_event_loop(loop)
loop.run_in_executor(None, update_contacts, {
'email': email,
'access_token': g.tokens['access_token']
})
Is there anything I need to do at the end to 'kill' the loop? What would be the proper way to close it, etc?
python python-3.x python-asyncio
python python-3.x python-asyncio
asked Nov 19 '18 at 4:20
David LDavid L
38516
38516
1
What you're designing there isfutures
. I believe you are usingThreadPoolExecutor
. You need to design it to be stoppable (eg. throw some flag/condition which can be interpreted for done) in order to finish and eventually let the daemon get shut down. Else, the event loop will live forever.
– Pruthvi Kumar
Nov 19 '18 at 6:15
1
Why are people lately hell-bent on usingasyncio
just to launch threads without anyasync
code? It is not whatasyncio
is there for, and using a plain Executor is much simpler.
– MisterMiyagi
Nov 19 '18 at 14:44
@MisterMiyagi perhaps you could show an example of how you'd suggest using it? I'm new to this all.
– David L
Nov 19 '18 at 19:46
add a comment |
1
What you're designing there isfutures
. I believe you are usingThreadPoolExecutor
. You need to design it to be stoppable (eg. throw some flag/condition which can be interpreted for done) in order to finish and eventually let the daemon get shut down. Else, the event loop will live forever.
– Pruthvi Kumar
Nov 19 '18 at 6:15
1
Why are people lately hell-bent on usingasyncio
just to launch threads without anyasync
code? It is not whatasyncio
is there for, and using a plain Executor is much simpler.
– MisterMiyagi
Nov 19 '18 at 14:44
@MisterMiyagi perhaps you could show an example of how you'd suggest using it? I'm new to this all.
– David L
Nov 19 '18 at 19:46
1
1
What you're designing there is
futures
. I believe you are using ThreadPoolExecutor
. You need to design it to be stoppable (eg. throw some flag/condition which can be interpreted for done) in order to finish and eventually let the daemon get shut down. Else, the event loop will live forever.– Pruthvi Kumar
Nov 19 '18 at 6:15
What you're designing there is
futures
. I believe you are using ThreadPoolExecutor
. You need to design it to be stoppable (eg. throw some flag/condition which can be interpreted for done) in order to finish and eventually let the daemon get shut down. Else, the event loop will live forever.– Pruthvi Kumar
Nov 19 '18 at 6:15
1
1
Why are people lately hell-bent on using
asyncio
just to launch threads without any async
code? It is not what asyncio
is there for, and using a plain Executor is much simpler.– MisterMiyagi
Nov 19 '18 at 14:44
Why are people lately hell-bent on using
asyncio
just to launch threads without any async
code? It is not what asyncio
is there for, and using a plain Executor is much simpler.– MisterMiyagi
Nov 19 '18 at 14:44
@MisterMiyagi perhaps you could show an example of how you'd suggest using it? I'm new to this all.
– David L
Nov 19 '18 at 19:46
@MisterMiyagi perhaps you could show an example of how you'd suggest using it? I'm new to this all.
– David L
Nov 19 '18 at 19:46
add a comment |
2 Answers
2
active
oldest
votes
You do not need to start any event loop in the first place. The concurrent.futures
package gives direct access to Executors, and threading
lets you launch individual Threads:
# raw thread
import threading
background_task = threading.Thread(
target=update_contacts, kwargs={
'email': email,
'access_token': g.tokens['access_token']
})
background_task.start()
# executor thread pool
from concurrent.futures import ThreadPoolExecutor
my_executor = ThreadPoolExecutor()
my_executor.submit(update_contacts, email=email, access_token=g.tokens['access_token'])
In general, a Thread
is simpler if you just want to launch a task and forget about it. A ThreadPoolExecutor
is more efficient if you have many small tasks at the same time; it can also be used to automatically wait for completion of several tasks.
print('start at', time.time())
with ThreadPoolExecutor() as executor:
executor.submit(time.sleep, 1)
executor.submit(time.sleep, 1)
executor.submit(time.sleep, 1)
executor.submit(time.sleep, 1)
print('done at', time.time()) # triggers after all 4 sleeps have finished
The primary purpose of loop.run_in_executor
is not to provide a ThreadPoolExecutor. It is meant to bridge the gap between Executors for blocking code and the event loop for non-blocking code. Without the later, there is no need to use asnycio
at all.
import time
import asyncio
def block(delay: float):
print("Stop! Blocking Time!")
time.sleep(delay) # block the current thread
print("Done! Blocking Time!")
async def nonblock(delay: float):
print("Erm.. Non-Blocking Time!")
await asyncio.sleep(delay)
print("Done! Non-Blocking Time!")
async def multiblock(delay: float):
loop = asyncio.get_event_loop()
await asyncio.gather( # await async natively and sync via executors
nonblock(delay),
loop.run_in_executor(None, block, delay),
nonblock(delay),
loop.run_in_executor(None, block, delay),
)
asyncio.run(multiblock(1))
add a comment |
Asyncio tasks can be canceled by calling the cancel
method on the Task
object. Tasks that run asynchronous code, such as those using the aiohttp library, will be canceled immediately. Tasks that run blocking code using run_in_executor
will not be canceled because they are run in an OS thread behind the scenes.
This is part of the reason why run_in_executor
is discouraged in asyncio code and is only intended as a stop-gap measure to include legacy blocking code in an asyncio program. (The other part is that the number of tasks is limited by the number of OS threads allowed by the pool, whereas the limit for the number of true asynchronous tasks is much higher.)
add a comment |
Your Answer
StackExchange.ifUsing("editor", function () {
StackExchange.using("externalEditor", function () {
StackExchange.using("snippets", function () {
StackExchange.snippets.init();
});
});
}, "code-snippets");
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "1"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53368265%2fhow-to-terminate-an-event-loop%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
2 Answers
2
active
oldest
votes
2 Answers
2
active
oldest
votes
active
oldest
votes
active
oldest
votes
You do not need to start any event loop in the first place. The concurrent.futures
package gives direct access to Executors, and threading
lets you launch individual Threads:
# raw thread
import threading
background_task = threading.Thread(
target=update_contacts, kwargs={
'email': email,
'access_token': g.tokens['access_token']
})
background_task.start()
# executor thread pool
from concurrent.futures import ThreadPoolExecutor
my_executor = ThreadPoolExecutor()
my_executor.submit(update_contacts, email=email, access_token=g.tokens['access_token'])
In general, a Thread
is simpler if you just want to launch a task and forget about it. A ThreadPoolExecutor
is more efficient if you have many small tasks at the same time; it can also be used to automatically wait for completion of several tasks.
print('start at', time.time())
with ThreadPoolExecutor() as executor:
executor.submit(time.sleep, 1)
executor.submit(time.sleep, 1)
executor.submit(time.sleep, 1)
executor.submit(time.sleep, 1)
print('done at', time.time()) # triggers after all 4 sleeps have finished
The primary purpose of loop.run_in_executor
is not to provide a ThreadPoolExecutor. It is meant to bridge the gap between Executors for blocking code and the event loop for non-blocking code. Without the later, there is no need to use asnycio
at all.
import time
import asyncio
def block(delay: float):
print("Stop! Blocking Time!")
time.sleep(delay) # block the current thread
print("Done! Blocking Time!")
async def nonblock(delay: float):
print("Erm.. Non-Blocking Time!")
await asyncio.sleep(delay)
print("Done! Non-Blocking Time!")
async def multiblock(delay: float):
loop = asyncio.get_event_loop()
await asyncio.gather( # await async natively and sync via executors
nonblock(delay),
loop.run_in_executor(None, block, delay),
nonblock(delay),
loop.run_in_executor(None, block, delay),
)
asyncio.run(multiblock(1))
add a comment |
You do not need to start any event loop in the first place. The concurrent.futures
package gives direct access to Executors, and threading
lets you launch individual Threads:
# raw thread
import threading
background_task = threading.Thread(
target=update_contacts, kwargs={
'email': email,
'access_token': g.tokens['access_token']
})
background_task.start()
# executor thread pool
from concurrent.futures import ThreadPoolExecutor
my_executor = ThreadPoolExecutor()
my_executor.submit(update_contacts, email=email, access_token=g.tokens['access_token'])
In general, a Thread
is simpler if you just want to launch a task and forget about it. A ThreadPoolExecutor
is more efficient if you have many small tasks at the same time; it can also be used to automatically wait for completion of several tasks.
print('start at', time.time())
with ThreadPoolExecutor() as executor:
executor.submit(time.sleep, 1)
executor.submit(time.sleep, 1)
executor.submit(time.sleep, 1)
executor.submit(time.sleep, 1)
print('done at', time.time()) # triggers after all 4 sleeps have finished
The primary purpose of loop.run_in_executor
is not to provide a ThreadPoolExecutor. It is meant to bridge the gap between Executors for blocking code and the event loop for non-blocking code. Without the later, there is no need to use asnycio
at all.
import time
import asyncio
def block(delay: float):
print("Stop! Blocking Time!")
time.sleep(delay) # block the current thread
print("Done! Blocking Time!")
async def nonblock(delay: float):
print("Erm.. Non-Blocking Time!")
await asyncio.sleep(delay)
print("Done! Non-Blocking Time!")
async def multiblock(delay: float):
loop = asyncio.get_event_loop()
await asyncio.gather( # await async natively and sync via executors
nonblock(delay),
loop.run_in_executor(None, block, delay),
nonblock(delay),
loop.run_in_executor(None, block, delay),
)
asyncio.run(multiblock(1))
add a comment |
You do not need to start any event loop in the first place. The concurrent.futures
package gives direct access to Executors, and threading
lets you launch individual Threads:
# raw thread
import threading
background_task = threading.Thread(
target=update_contacts, kwargs={
'email': email,
'access_token': g.tokens['access_token']
})
background_task.start()
# executor thread pool
from concurrent.futures import ThreadPoolExecutor
my_executor = ThreadPoolExecutor()
my_executor.submit(update_contacts, email=email, access_token=g.tokens['access_token'])
In general, a Thread
is simpler if you just want to launch a task and forget about it. A ThreadPoolExecutor
is more efficient if you have many small tasks at the same time; it can also be used to automatically wait for completion of several tasks.
print('start at', time.time())
with ThreadPoolExecutor() as executor:
executor.submit(time.sleep, 1)
executor.submit(time.sleep, 1)
executor.submit(time.sleep, 1)
executor.submit(time.sleep, 1)
print('done at', time.time()) # triggers after all 4 sleeps have finished
The primary purpose of loop.run_in_executor
is not to provide a ThreadPoolExecutor. It is meant to bridge the gap between Executors for blocking code and the event loop for non-blocking code. Without the later, there is no need to use asnycio
at all.
import time
import asyncio
def block(delay: float):
print("Stop! Blocking Time!")
time.sleep(delay) # block the current thread
print("Done! Blocking Time!")
async def nonblock(delay: float):
print("Erm.. Non-Blocking Time!")
await asyncio.sleep(delay)
print("Done! Non-Blocking Time!")
async def multiblock(delay: float):
loop = asyncio.get_event_loop()
await asyncio.gather( # await async natively and sync via executors
nonblock(delay),
loop.run_in_executor(None, block, delay),
nonblock(delay),
loop.run_in_executor(None, block, delay),
)
asyncio.run(multiblock(1))
You do not need to start any event loop in the first place. The concurrent.futures
package gives direct access to Executors, and threading
lets you launch individual Threads:
# raw thread
import threading
background_task = threading.Thread(
target=update_contacts, kwargs={
'email': email,
'access_token': g.tokens['access_token']
})
background_task.start()
# executor thread pool
from concurrent.futures import ThreadPoolExecutor
my_executor = ThreadPoolExecutor()
my_executor.submit(update_contacts, email=email, access_token=g.tokens['access_token'])
In general, a Thread
is simpler if you just want to launch a task and forget about it. A ThreadPoolExecutor
is more efficient if you have many small tasks at the same time; it can also be used to automatically wait for completion of several tasks.
print('start at', time.time())
with ThreadPoolExecutor() as executor:
executor.submit(time.sleep, 1)
executor.submit(time.sleep, 1)
executor.submit(time.sleep, 1)
executor.submit(time.sleep, 1)
print('done at', time.time()) # triggers after all 4 sleeps have finished
The primary purpose of loop.run_in_executor
is not to provide a ThreadPoolExecutor. It is meant to bridge the gap between Executors for blocking code and the event loop for non-blocking code. Without the later, there is no need to use asnycio
at all.
import time
import asyncio
def block(delay: float):
print("Stop! Blocking Time!")
time.sleep(delay) # block the current thread
print("Done! Blocking Time!")
async def nonblock(delay: float):
print("Erm.. Non-Blocking Time!")
await asyncio.sleep(delay)
print("Done! Non-Blocking Time!")
async def multiblock(delay: float):
loop = asyncio.get_event_loop()
await asyncio.gather( # await async natively and sync via executors
nonblock(delay),
loop.run_in_executor(None, block, delay),
nonblock(delay),
loop.run_in_executor(None, block, delay),
)
asyncio.run(multiblock(1))
edited Nov 19 '18 at 20:47
answered Nov 19 '18 at 20:24
MisterMiyagiMisterMiyagi
7,7022243
7,7022243
add a comment |
add a comment |
Asyncio tasks can be canceled by calling the cancel
method on the Task
object. Tasks that run asynchronous code, such as those using the aiohttp library, will be canceled immediately. Tasks that run blocking code using run_in_executor
will not be canceled because they are run in an OS thread behind the scenes.
This is part of the reason why run_in_executor
is discouraged in asyncio code and is only intended as a stop-gap measure to include legacy blocking code in an asyncio program. (The other part is that the number of tasks is limited by the number of OS threads allowed by the pool, whereas the limit for the number of true asynchronous tasks is much higher.)
add a comment |
Asyncio tasks can be canceled by calling the cancel
method on the Task
object. Tasks that run asynchronous code, such as those using the aiohttp library, will be canceled immediately. Tasks that run blocking code using run_in_executor
will not be canceled because they are run in an OS thread behind the scenes.
This is part of the reason why run_in_executor
is discouraged in asyncio code and is only intended as a stop-gap measure to include legacy blocking code in an asyncio program. (The other part is that the number of tasks is limited by the number of OS threads allowed by the pool, whereas the limit for the number of true asynchronous tasks is much higher.)
add a comment |
Asyncio tasks can be canceled by calling the cancel
method on the Task
object. Tasks that run asynchronous code, such as those using the aiohttp library, will be canceled immediately. Tasks that run blocking code using run_in_executor
will not be canceled because they are run in an OS thread behind the scenes.
This is part of the reason why run_in_executor
is discouraged in asyncio code and is only intended as a stop-gap measure to include legacy blocking code in an asyncio program. (The other part is that the number of tasks is limited by the number of OS threads allowed by the pool, whereas the limit for the number of true asynchronous tasks is much higher.)
Asyncio tasks can be canceled by calling the cancel
method on the Task
object. Tasks that run asynchronous code, such as those using the aiohttp library, will be canceled immediately. Tasks that run blocking code using run_in_executor
will not be canceled because they are run in an OS thread behind the scenes.
This is part of the reason why run_in_executor
is discouraged in asyncio code and is only intended as a stop-gap measure to include legacy blocking code in an asyncio program. (The other part is that the number of tasks is limited by the number of OS threads allowed by the pool, whereas the limit for the number of true asynchronous tasks is much higher.)
edited Nov 19 '18 at 14:49
answered Nov 19 '18 at 14:41
user4815162342user4815162342
62.2k593145
62.2k593145
add a comment |
add a comment |
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53368265%2fhow-to-terminate-an-event-loop%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
1
What you're designing there is
futures
. I believe you are usingThreadPoolExecutor
. You need to design it to be stoppable (eg. throw some flag/condition which can be interpreted for done) in order to finish and eventually let the daemon get shut down. Else, the event loop will live forever.– Pruthvi Kumar
Nov 19 '18 at 6:15
1
Why are people lately hell-bent on using
asyncio
just to launch threads without anyasync
code? It is not whatasyncio
is there for, and using a plain Executor is much simpler.– MisterMiyagi
Nov 19 '18 at 14:44
@MisterMiyagi perhaps you could show an example of how you'd suggest using it? I'm new to this all.
– David L
Nov 19 '18 at 19:46