Using Julia package manager offline
We recently decided to make Julia Language available on our cluster systems. The cluster system is not able to connect to the internet.
Is there any way to download all Julia packages and make them available for our different users to install and use them offline?
Another option that we have is a system that can connect to the internet temporarily, but it is always connected to the main cluster system. Is there any way to use this system as a mirror for the Julia packages or not?
We want to use "Julia 1.0.1".
our cluster operation system is: "CentOS 5.5
notes: I have seen the question asked before here, but it is for Julia 0.6 and a single package that will be copied by hand. I want that user uses the Pkg.add <pkgName> command but instead of the internet, the package manager gets the packages from our offline system.
Thank you for your help and time.
julia
asked Nov 20 '18 at 14:04
Alireza Ghavami NiaAlireza Ghavami Nia
969
|
show 4 more comments
We recently decided to make Julia Language available on our cluster systems. The cluster system is not able to connect to the internet.
Is there any way to download all Julia packages and make them available for our different users to install and use them offline?
Another option that we have is a system that can connect to the internet temporarily, but it is always connected to the main cluster system. Is there any way to use this system as a mirror for the Julia packages or not?
We want to use "Julia 1.0.1".
our cluster operation system is: "CentOS 5.5
notes: I have seen the question asked before here, but it is for Julia 0.6 and a single package that will be copied by hand. I want that user uses the Pkg.add <pkgName> command but instead of the internet, the package manager gets the packages from our offline system.
Thank you for your help and time.
julia
asked Nov 20 '18 at 14:04
Alireza Ghavami NiaAlireza Ghavami Nia
969
xref: discourse.julialang.org/t/using-julia-package-manager-offline/…
– crstnbr
Nov 20 '18 at 14:22
1
in Julia it is easy to reattach all packages via theJULIA_DEPOT_PATH. If you have Internet on one node and all nodes are identical you could use it as "installation & compilation node" and then replicate the contents to other servers in your cluster.
– Przemyslaw Szufel
Nov 24 '18 at 1:12
1
I would try This: 1. Attach machine A to the internet. 2.export JULIA_DEPOT_PATH="/my/package/folder". 3.julia. 4. Install whatever you need. 5. In the Julia package manager run(v1.0) pkg> precompileto precompile everything. 6. Copy "/my/package/folder" to machine B (without internet). 7. Run on machine B:export JULIA_DEPOT_PATH="/my/package/folder"before runningjulia. If machine A and B have the same hardware configuration this should work.
– Przemyslaw Szufel
Nov 24 '18 at 17:12
1
As far as I know Julia usesJULIA_DEPOT_PATHto store all compiled module files. As long as they are not compiled (that no new modules are added to Julia installation) it should be possible to share them in read-only mode across many users (I have not tested in though).
– Przemyslaw Szufel
Nov 27 '18 at 19:24
1
Now regarding the size: the main reason for huge size ofJULIA_DEPOT_PATHis the Python installation. Bu here it is easy. Simply do not use the inbuilt Python!. This can be achieved by setting ENV["PYTHON"] Julia variable before running Pkg.add("PyCall"). This will allow you to use an external Python Anaconda and I am pretty sure there are already many recipes how to share that across large user bases.
– Przemyslaw Szufel
Nov 27 '18 at 19:26
|
show 4 more comments
We recently decided to make Julia Language available on our cluster systems. The cluster system is not able to connect to the internet.
Is there any way to download all Julia packages and make them available for our different users to install and use them offline?
Another option that we have is a system that can connect to the internet temporarily, but it is always connected to the main cluster system. Is there any way to use this system as a mirror for the Julia packages or not?
We want to use "Julia 1.0.1".
our cluster operation system is: "CentOS 5.5
notes: I have seen the question asked before here, but it is for Julia 0.6 and a single package that will be copied by hand. I want that user uses the Pkg.add <pkgName> command but instead of the internet, the package manager gets the packages from our offline system.
Thank you for your help and time.
julia
asked Nov 20 '18 at 14:04
Alireza Ghavami NiaAlireza Ghavami Nia
969
We recently decided to make Julia Language available on our cluster systems. The cluster system is not able to connect to the internet.
Is there any way to download all Julia packages and make them available for our different users to install and use them offline?
Another option that we have is a system that can connect to the internet temporarily, but it is always connected to the main cluster system. Is there any way to use this system as a mirror for the Julia packages or not?
We want to use "Julia 1.0.1".
our cluster operation system is: "CentOS 5.5
notes: I have seen the question asked before here, but it is for Julia 0.6 and a single package that will be copied by hand. I want that user uses the Pkg.add <pkgName> command but instead of the internet, the package manager gets the packages from our offline system.
Thank you for your help and time.
julia
julia
asked Nov 20 '18 at 14:04
Alireza Ghavami NiaAlireza Ghavami Nia
969
asked Nov 20 '18 at 14:04
Alireza Ghavami NiaAlireza Ghavami Nia
969
asked Nov 20 '18 at 14:04
Alireza Ghavami NiaAlireza Ghavami Nia
969
asked Nov 20 '18 at 14:04
Alireza Ghavami NiaAlireza Ghavami Nia
969
asked Nov 20 '18 at 14:04
Alireza Ghavami NiaAlireza Ghavami Nia
969
969
xref: discourse.julialang.org/t/using-julia-package-manager-offline/…
– crstnbr
Nov 20 '18 at 14:22
1
in Julia it is easy to reattach all packages via theJULIA_DEPOT_PATH. If you have Internet on one node and all nodes are identical you could use it as "installation & compilation node" and then replicate the contents to other servers in your cluster.
– Przemyslaw Szufel
Nov 24 '18 at 1:12
1
I would try This: 1. Attach machine A to the internet. 2.export JULIA_DEPOT_PATH="/my/package/folder". 3.julia. 4. Install whatever you need. 5. In the Julia package manager run(v1.0) pkg> precompileto precompile everything. 6. Copy "/my/package/folder" to machine B (without internet). 7. Run on machine B:export JULIA_DEPOT_PATH="/my/package/folder"before runningjulia. If machine A and B have the same hardware configuration this should work.
– Przemyslaw Szufel
Nov 24 '18 at 17:12
1
As far as I know Julia usesJULIA_DEPOT_PATHto store all compiled module files. As long as they are not compiled (that no new modules are added to Julia installation) it should be possible to share them in read-only mode across many users (I have not tested in though).
– Przemyslaw Szufel
Nov 27 '18 at 19:24
1
Now regarding the size: the main reason for huge size ofJULIA_DEPOT_PATHis the Python installation. Bu here it is easy. Simply do not use the inbuilt Python!. This can be achieved by setting ENV["PYTHON"] Julia variable before running Pkg.add("PyCall"). This will allow you to use an external Python Anaconda and I am pretty sure there are already many recipes how to share that across large user bases.
– Przemyslaw Szufel
Nov 27 '18 at 19:26
|
show 4 more comments
xref: discourse.julialang.org/t/using-julia-package-manager-offline/…
– crstnbr
Nov 20 '18 at 14:22
1
in Julia it is easy to reattach all packages via theJULIA_DEPOT_PATH. If you have Internet on one node and all nodes are identical you could use it as "installation & compilation node" and then replicate the contents to other servers in your cluster.
– Przemyslaw Szufel
Nov 24 '18 at 1:12
1
I would try This: 1. Attach machine A to the internet. 2.export JULIA_DEPOT_PATH="/my/package/folder". 3.julia. 4. Install whatever you need. 5. In the Julia package manager run(v1.0) pkg> precompileto precompile everything. 6. Copy "/my/package/folder" to machine B (without internet). 7. Run on machine B:export JULIA_DEPOT_PATH="/my/package/folder"before runningjulia. If machine A and B have the same hardware configuration this should work.
– Przemyslaw Szufel
Nov 24 '18 at 17:12
1
As far as I know Julia usesJULIA_DEPOT_PATHto store all compiled module files. As long as they are not compiled (that no new modules are added to Julia installation) it should be possible to share them in read-only mode across many users (I have not tested in though).
– Przemyslaw Szufel
Nov 27 '18 at 19:24
1
Now regarding the size: the main reason for huge size ofJULIA_DEPOT_PATHis the Python installation. Bu here it is easy. Simply do not use the inbuilt Python!. This can be achieved by setting ENV["PYTHON"] Julia variable before running Pkg.add("PyCall"). This will allow you to use an external Python Anaconda and I am pretty sure there are already many recipes how to share that across large user bases.
– Przemyslaw Szufel
Nov 27 '18 at 19:26
xref: discourse.julialang.org/t/using-julia-package-manager-offline/…
– crstnbr
Nov 20 '18 at 14:22
xref: discourse.julialang.org/t/using-julia-package-manager-offline/…
– crstnbr
Nov 20 '18 at 14:22
1
1
in Julia it is easy to reattach all packages via the
JULIA_DEPOT_PATH. If you have Internet on one node and all nodes are identical you could use it as "installation & compilation node" and then replicate the contents to other servers in your cluster.– Przemyslaw Szufel
Nov 24 '18 at 1:12
in Julia it is easy to reattach all packages via the
JULIA_DEPOT_PATH. If you have Internet on one node and all nodes are identical you could use it as "installation & compilation node" and then replicate the contents to other servers in your cluster.– Przemyslaw Szufel
Nov 24 '18 at 1:12
1
1
I would try This: 1. Attach machine A to the internet. 2.
export JULIA_DEPOT_PATH="/my/package/folder". 3. julia. 4. Install whatever you need. 5. In the Julia package manager run (v1.0) pkg> precompile to precompile everything. 6. Copy "/my/package/folder" to machine B (without internet). 7. Run on machine B: export JULIA_DEPOT_PATH="/my/package/folder" before running julia. If machine A and B have the same hardware configuration this should work.– Przemyslaw Szufel
Nov 24 '18 at 17:12
I would try This: 1. Attach machine A to the internet. 2.
export JULIA_DEPOT_PATH="/my/package/folder". 3. julia. 4. Install whatever you need. 5. In the Julia package manager run (v1.0) pkg> precompile to precompile everything. 6. Copy "/my/package/folder" to machine B (without internet). 7. Run on machine B: export JULIA_DEPOT_PATH="/my/package/folder" before running julia. If machine A and B have the same hardware configuration this should work.– Przemyslaw Szufel
Nov 24 '18 at 17:12
1
1
As far as I know Julia uses
JULIA_DEPOT_PATH to store all compiled module files. As long as they are not compiled (that no new modules are added to Julia installation) it should be possible to share them in read-only mode across many users (I have not tested in though).– Przemyslaw Szufel
Nov 27 '18 at 19:24
As far as I know Julia uses
JULIA_DEPOT_PATH to store all compiled module files. As long as they are not compiled (that no new modules are added to Julia installation) it should be possible to share them in read-only mode across many users (I have not tested in though).– Przemyslaw Szufel
Nov 27 '18 at 19:24
1
1
Now regarding the size: the main reason for huge size of
JULIA_DEPOT_PATH is the Python installation. Bu here it is easy. Simply do not use the inbuilt Python!. This can be achieved by setting ENV["PYTHON"] Julia variable before running Pkg.add("PyCall"). This will allow you to use an external Python Anaconda and I am pretty sure there are already many recipes how to share that across large user bases.– Przemyslaw Szufel
Nov 27 '18 at 19:26
Now regarding the size: the main reason for huge size of
JULIA_DEPOT_PATH is the Python installation. Bu here it is easy. Simply do not use the inbuilt Python!. This can be achieved by setting ENV["PYTHON"] Julia variable before running Pkg.add("PyCall"). This will allow you to use an external Python Anaconda and I am pretty sure there are already many recipes how to share that across large user bases.– Przemyslaw Szufel
Nov 27 '18 at 19:26
|
show 4 more comments
0
active
oldest
votes
Your Answer
StackExchange.ifUsing("editor", function () {
StackExchange.using("externalEditor", function () {
StackExchange.using("snippets", function () {
StackExchange.snippets.init();
});
});
}, "code-snippets");
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "1"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53394757%2fusing-julia-package-manager-offline%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
0
active
oldest
votes
0
active
oldest
votes
active
oldest
votes
active
oldest
votes
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53394757%2fusing-julia-package-manager-offline%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
xref: discourse.julialang.org/t/using-julia-package-manager-offline/…
– crstnbr
Nov 20 '18 at 14:22
1
in Julia it is easy to reattach all packages via the
JULIA_DEPOT_PATH. If you have Internet on one node and all nodes are identical you could use it as "installation & compilation node" and then replicate the contents to other servers in your cluster.– Przemyslaw Szufel
Nov 24 '18 at 1:12
1
I would try This: 1. Attach machine A to the internet. 2.
export JULIA_DEPOT_PATH="/my/package/folder". 3.julia. 4. Install whatever you need. 5. In the Julia package manager run(v1.0) pkg> precompileto precompile everything. 6. Copy "/my/package/folder" to machine B (without internet). 7. Run on machine B:export JULIA_DEPOT_PATH="/my/package/folder"before runningjulia. If machine A and B have the same hardware configuration this should work.– Przemyslaw Szufel
Nov 24 '18 at 17:12
1
As far as I know Julia uses
JULIA_DEPOT_PATHto store all compiled module files. As long as they are not compiled (that no new modules are added to Julia installation) it should be possible to share them in read-only mode across many users (I have not tested in though).– Przemyslaw Szufel
Nov 27 '18 at 19:24
1
Now regarding the size: the main reason for huge size of
JULIA_DEPOT_PATHis the Python installation. Bu here it is easy. Simply do not use the inbuilt Python!. This can be achieved by setting ENV["PYTHON"] Julia variable before running Pkg.add("PyCall"). This will allow you to use an external Python Anaconda and I am pretty sure there are already many recipes how to share that across large user bases.– Przemyslaw Szufel
Nov 27 '18 at 19:26