Use zipped virtualenv
0
I'd like to create a virtual environment, zip it up, and execute code using the zipped virtual environment on another machine; it seems to be possible based on this article.
However, I'm having trouble and sourcing a virtualenv even locally using the technique described in the article.
$ conda create -y -n my_test python=3.5 numpy pandas
$ zip -r env.zip ~/anaconda3/envs/my_test
$ source env.zip/lib/python3.5/venv/scripts/common/activate
-bash: env.zip/lib/python3.5/venv/scripts/common/activate: Not a directory
$ tar cvfhz env.tgz ~/anaconda3/envs/my_test
$ source env.tgz/lib/python3.5/venv/scripts/common/activate
-bash: env.tgz/lib/python3.5/venv/scripts/common/activate: Not a directory
I've tried this on Mac OS X 10.13.6 and Ubuntu 16.04.
python bash pip virtualenv conda
asked Nov 20 '18 at 16:02
Michael KMichael K
67521539
|
show 1 more comment
0
I'd like to create a virtual environment, zip it up, and execute code using the zipped virtual environment on another machine; it seems to be possible based on this article.
However, I'm having trouble and sourcing a virtualenv even locally using the technique described in the article.
$ conda create -y -n my_test python=3.5 numpy pandas
$ zip -r env.zip ~/anaconda3/envs/my_test
$ source env.zip/lib/python3.5/venv/scripts/common/activate
-bash: env.zip/lib/python3.5/venv/scripts/common/activate: Not a directory
$ tar cvfhz env.tgz ~/anaconda3/envs/my_test
$ source env.tgz/lib/python3.5/venv/scripts/common/activate
-bash: env.tgz/lib/python3.5/venv/scripts/common/activate: Not a directory
I've tried this on Mac OS X 10.13.6 and Ubuntu 16.04.
python bash pip virtualenv conda
asked Nov 20 '18 at 16:02
Michael KMichael K
67521539
4
virtualenv's aren't portable and they aren't distribution/deployment mechanisms. Recreate a venv in the target directory/host or use a different distribution/deployment.
– phd
Nov 20 '18 at 17:04
3
Adding to what @phd posted - the source command you execute will not work; you can't treat a zip or tar archive as part of the filesystem. The pathmyarchive.zip/somescriptis thus invalid. It works for the author of the article you posted only because Hive will automatically unpack the archives added viaADD ARCHIVE. However, even after extraction, the venv will be usable only in identical setups on identical machines and when put in the same directory; it's cheaper just to create a new venv.
– hoefling
Nov 20 '18 at 17:19
Got it, thank you.
– Michael K
Nov 20 '18 at 17:35
1
The zipped packages might work , You have to copy those zipped packages to python local site-packages folder but certain packages like pandas which depends onCthrows error depends on the OS build. Like pandas compiled on ubuntu won't work on AWS EMR machines with Cent OS.
– Yash Kumar Atri
Nov 21 '18 at 10:29
1
As far as I understand pex does pretty much this.
– syntonym
Feb 26 at 22:57
|
show 1 more comment
0
0
0
I'd like to create a virtual environment, zip it up, and execute code using the zipped virtual environment on another machine; it seems to be possible based on this article.
However, I'm having trouble and sourcing a virtualenv even locally using the technique described in the article.
$ conda create -y -n my_test python=3.5 numpy pandas
$ zip -r env.zip ~/anaconda3/envs/my_test
$ source env.zip/lib/python3.5/venv/scripts/common/activate
-bash: env.zip/lib/python3.5/venv/scripts/common/activate: Not a directory
$ tar cvfhz env.tgz ~/anaconda3/envs/my_test
$ source env.tgz/lib/python3.5/venv/scripts/common/activate
-bash: env.tgz/lib/python3.5/venv/scripts/common/activate: Not a directory
I've tried this on Mac OS X 10.13.6 and Ubuntu 16.04.
python bash pip virtualenv conda
asked Nov 20 '18 at 16:02
Michael KMichael K
67521539
I'd like to create a virtual environment, zip it up, and execute code using the zipped virtual environment on another machine; it seems to be possible based on this article.
However, I'm having trouble and sourcing a virtualenv even locally using the technique described in the article.
$ conda create -y -n my_test python=3.5 numpy pandas
$ zip -r env.zip ~/anaconda3/envs/my_test
$ source env.zip/lib/python3.5/venv/scripts/common/activate
-bash: env.zip/lib/python3.5/venv/scripts/common/activate: Not a directory
$ tar cvfhz env.tgz ~/anaconda3/envs/my_test
$ source env.tgz/lib/python3.5/venv/scripts/common/activate
-bash: env.tgz/lib/python3.5/venv/scripts/common/activate: Not a directory
I've tried this on Mac OS X 10.13.6 and Ubuntu 16.04.
python bash pip virtualenv conda
python bash pip virtualenv conda
asked Nov 20 '18 at 16:02
Michael KMichael K
67521539
asked Nov 20 '18 at 16:02
Michael KMichael K
67521539
asked Nov 20 '18 at 16:02
Michael KMichael K
67521539
asked Nov 20 '18 at 16:02
Michael KMichael K
67521539
asked Nov 20 '18 at 16:02
Michael KMichael K
67521539
67521539
4
virtualenv's aren't portable and they aren't distribution/deployment mechanisms. Recreate a venv in the target directory/host or use a different distribution/deployment.
– phd
Nov 20 '18 at 17:04
3
Adding to what @phd posted - the source command you execute will not work; you can't treat a zip or tar archive as part of the filesystem. The pathmyarchive.zip/somescriptis thus invalid. It works for the author of the article you posted only because Hive will automatically unpack the archives added viaADD ARCHIVE. However, even after extraction, the venv will be usable only in identical setups on identical machines and when put in the same directory; it's cheaper just to create a new venv.
– hoefling
Nov 20 '18 at 17:19
Got it, thank you.
– Michael K
Nov 20 '18 at 17:35
1
The zipped packages might work , You have to copy those zipped packages to python local site-packages folder but certain packages like pandas which depends onCthrows error depends on the OS build. Like pandas compiled on ubuntu won't work on AWS EMR machines with Cent OS.
– Yash Kumar Atri
Nov 21 '18 at 10:29
1
As far as I understand pex does pretty much this.
– syntonym
Feb 26 at 22:57
|
show 1 more comment
4
virtualenv's aren't portable and they aren't distribution/deployment mechanisms. Recreate a venv in the target directory/host or use a different distribution/deployment.
– phd
Nov 20 '18 at 17:04
3
Adding to what @phd posted - the source command you execute will not work; you can't treat a zip or tar archive as part of the filesystem. The pathmyarchive.zip/somescriptis thus invalid. It works for the author of the article you posted only because Hive will automatically unpack the archives added viaADD ARCHIVE. However, even after extraction, the venv will be usable only in identical setups on identical machines and when put in the same directory; it's cheaper just to create a new venv.
– hoefling
Nov 20 '18 at 17:19
Got it, thank you.
– Michael K
Nov 20 '18 at 17:35
1
The zipped packages might work , You have to copy those zipped packages to python local site-packages folder but certain packages like pandas which depends onCthrows error depends on the OS build. Like pandas compiled on ubuntu won't work on AWS EMR machines with Cent OS.
– Yash Kumar Atri
Nov 21 '18 at 10:29
1
As far as I understand pex does pretty much this.
– syntonym
Feb 26 at 22:57
4
4
virtualenv's aren't portable and they aren't distribution/deployment mechanisms. Recreate a venv in the target directory/host or use a different distribution/deployment.
– phd
Nov 20 '18 at 17:04
virtualenv's aren't portable and they aren't distribution/deployment mechanisms. Recreate a venv in the target directory/host or use a different distribution/deployment.
– phd
Nov 20 '18 at 17:04
3
3
Adding to what @phd posted - the source command you execute will not work; you can't treat a zip or tar archive as part of the filesystem. The path
myarchive.zip/somescript is thus invalid. It works for the author of the article you posted only because Hive will automatically unpack the archives added via ADD ARCHIVE. However, even after extraction, the venv will be usable only in identical setups on identical machines and when put in the same directory; it's cheaper just to create a new venv.– hoefling
Nov 20 '18 at 17:19
Adding to what @phd posted - the source command you execute will not work; you can't treat a zip or tar archive as part of the filesystem. The path
myarchive.zip/somescript is thus invalid. It works for the author of the article you posted only because Hive will automatically unpack the archives added via ADD ARCHIVE. However, even after extraction, the venv will be usable only in identical setups on identical machines and when put in the same directory; it's cheaper just to create a new venv.– hoefling
Nov 20 '18 at 17:19
Got it, thank you.
– Michael K
Nov 20 '18 at 17:35
Got it, thank you.
– Michael K
Nov 20 '18 at 17:35
1
1
The zipped packages might work , You have to copy those zipped packages to python local site-packages folder but certain packages like pandas which depends on
C throws error depends on the OS build. Like pandas compiled on ubuntu won't work on AWS EMR machines with Cent OS.– Yash Kumar Atri
Nov 21 '18 at 10:29
The zipped packages might work , You have to copy those zipped packages to python local site-packages folder but certain packages like pandas which depends on
C throws error depends on the OS build. Like pandas compiled on ubuntu won't work on AWS EMR machines with Cent OS.– Yash Kumar Atri
Nov 21 '18 at 10:29
1
1
As far as I understand pex does pretty much this.
– syntonym
Feb 26 at 22:57
As far as I understand pex does pretty much this.
– syntonym
Feb 26 at 22:57
|
show 1 more comment
0
active
oldest
votes
Your Answer
StackExchange.ifUsing("editor", function () {
StackExchange.using("externalEditor", function () {
StackExchange.using("snippets", function () {
StackExchange.snippets.init();
});
});
}, "code-snippets");
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "1"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
draft saved
draft discarded
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53396947%2fuse-zipped-virtualenv%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
0
active
oldest
votes
0
active
oldest
votes
active
oldest
votes
active
oldest
votes
draft saved
draft discarded
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
draft saved
draft discarded
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53396947%2fuse-zipped-virtualenv%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
4
virtualenv's aren't portable and they aren't distribution/deployment mechanisms. Recreate a venv in the target directory/host or use a different distribution/deployment.
– phd
Nov 20 '18 at 17:04
3
Adding to what @phd posted - the source command you execute will not work; you can't treat a zip or tar archive as part of the filesystem. The path
myarchive.zip/somescriptis thus invalid. It works for the author of the article you posted only because Hive will automatically unpack the archives added viaADD ARCHIVE. However, even after extraction, the venv will be usable only in identical setups on identical machines and when put in the same directory; it's cheaper just to create a new venv.– hoefling
Nov 20 '18 at 17:19
Got it, thank you.
– Michael K
Nov 20 '18 at 17:35
1
The zipped packages might work , You have to copy those zipped packages to python local site-packages folder but certain packages like pandas which depends on
Cthrows error depends on the OS build. Like pandas compiled on ubuntu won't work on AWS EMR machines with Cent OS.– Yash Kumar Atri
Nov 21 '18 at 10:29
1
As far as I understand pex does pretty much this.
– syntonym
Feb 26 at 22:57