Resample or aggregate large raster limiting memory use R












0















I'm trying to resample a huge worldpop population raster but I keep crashing my linux instance of R which has 32GB memory. When I launch a 52gb memory google compute instance, the code below works, but it crashes my regular 32gb ram computer.



Is there a way to do either a raster aggregation or resampling LIMITING MEMORY USE?



Download code for a large worldpop raster that I am having issues resampling:



###download the huge raster to recreate scale problem
devtools::install_github("nbarsch/spaceheater")
library(spaceheater)
getWPdownload("Tanzania","Population","adj",2015) #warning: downloads near 1gb file
library(raster)
wpras <- raster("TANZANIA_Population_adj_2015.tif")


TWO METHODS that work on a computer with 52GB ram, but kill my local computer 32GB ram:



#aggregate method
agras <- raster::aggregate(wpras,fact=10,expand=T)
#returns "Killed"

#resamp method
reras <- raster(nrow = ceiling(nrow(wpras)/10), ncol = ceiling(ncol(wpras)/10))
reras2 <- raster::resample(wpras, reras, method="bilinear")
#returns Error: cannot allocate vector of size 1.3 Gb


Anyone have a solution that doesn't use all the ram? Thanks!










share|improve this question























  • # increase ram memory.limit()

    – Dr. Flow
    Nov 16 '18 at 10:42
















0















I'm trying to resample a huge worldpop population raster but I keep crashing my linux instance of R which has 32GB memory. When I launch a 52gb memory google compute instance, the code below works, but it crashes my regular 32gb ram computer.



Is there a way to do either a raster aggregation or resampling LIMITING MEMORY USE?



Download code for a large worldpop raster that I am having issues resampling:



###download the huge raster to recreate scale problem
devtools::install_github("nbarsch/spaceheater")
library(spaceheater)
getWPdownload("Tanzania","Population","adj",2015) #warning: downloads near 1gb file
library(raster)
wpras <- raster("TANZANIA_Population_adj_2015.tif")


TWO METHODS that work on a computer with 52GB ram, but kill my local computer 32GB ram:



#aggregate method
agras <- raster::aggregate(wpras,fact=10,expand=T)
#returns "Killed"

#resamp method
reras <- raster(nrow = ceiling(nrow(wpras)/10), ncol = ceiling(ncol(wpras)/10))
reras2 <- raster::resample(wpras, reras, method="bilinear")
#returns Error: cannot allocate vector of size 1.3 Gb


Anyone have a solution that doesn't use all the ram? Thanks!










share|improve this question























  • # increase ram memory.limit()

    – Dr. Flow
    Nov 16 '18 at 10:42














0












0








0








I'm trying to resample a huge worldpop population raster but I keep crashing my linux instance of R which has 32GB memory. When I launch a 52gb memory google compute instance, the code below works, but it crashes my regular 32gb ram computer.



Is there a way to do either a raster aggregation or resampling LIMITING MEMORY USE?



Download code for a large worldpop raster that I am having issues resampling:



###download the huge raster to recreate scale problem
devtools::install_github("nbarsch/spaceheater")
library(spaceheater)
getWPdownload("Tanzania","Population","adj",2015) #warning: downloads near 1gb file
library(raster)
wpras <- raster("TANZANIA_Population_adj_2015.tif")


TWO METHODS that work on a computer with 52GB ram, but kill my local computer 32GB ram:



#aggregate method
agras <- raster::aggregate(wpras,fact=10,expand=T)
#returns "Killed"

#resamp method
reras <- raster(nrow = ceiling(nrow(wpras)/10), ncol = ceiling(ncol(wpras)/10))
reras2 <- raster::resample(wpras, reras, method="bilinear")
#returns Error: cannot allocate vector of size 1.3 Gb


Anyone have a solution that doesn't use all the ram? Thanks!










share|improve this question














I'm trying to resample a huge worldpop population raster but I keep crashing my linux instance of R which has 32GB memory. When I launch a 52gb memory google compute instance, the code below works, but it crashes my regular 32gb ram computer.



Is there a way to do either a raster aggregation or resampling LIMITING MEMORY USE?



Download code for a large worldpop raster that I am having issues resampling:



###download the huge raster to recreate scale problem
devtools::install_github("nbarsch/spaceheater")
library(spaceheater)
getWPdownload("Tanzania","Population","adj",2015) #warning: downloads near 1gb file
library(raster)
wpras <- raster("TANZANIA_Population_adj_2015.tif")


TWO METHODS that work on a computer with 52GB ram, but kill my local computer 32GB ram:



#aggregate method
agras <- raster::aggregate(wpras,fact=10,expand=T)
#returns "Killed"

#resamp method
reras <- raster(nrow = ceiling(nrow(wpras)/10), ncol = ceiling(ncol(wpras)/10))
reras2 <- raster::resample(wpras, reras, method="bilinear")
#returns Error: cannot allocate vector of size 1.3 Gb


Anyone have a solution that doesn't use all the ram? Thanks!







r memory raster ram






share|improve this question













share|improve this question











share|improve this question




share|improve this question










asked Nov 16 '18 at 8:46









Neal BarschNeal Barsch

615114




615114













  • # increase ram memory.limit()

    – Dr. Flow
    Nov 16 '18 at 10:42



















  • # increase ram memory.limit()

    – Dr. Flow
    Nov 16 '18 at 10:42

















# increase ram memory.limit()

– Dr. Flow
Nov 16 '18 at 10:42





# increase ram memory.limit()

– Dr. Flow
Nov 16 '18 at 10:42












1 Answer
1






active

oldest

votes


















0














You probably should update the raster package. The previous short-lived release (2.7-15) had an error in the memory settings. Version 2.8-4 should have fixed that.






share|improve this answer























    Your Answer






    StackExchange.ifUsing("editor", function () {
    StackExchange.using("externalEditor", function () {
    StackExchange.using("snippets", function () {
    StackExchange.snippets.init();
    });
    });
    }, "code-snippets");

    StackExchange.ready(function() {
    var channelOptions = {
    tags: "".split(" "),
    id: "1"
    };
    initTagRenderer("".split(" "), "".split(" "), channelOptions);

    StackExchange.using("externalEditor", function() {
    // Have to fire editor after snippets, if snippets enabled
    if (StackExchange.settings.snippets.snippetsEnabled) {
    StackExchange.using("snippets", function() {
    createEditor();
    });
    }
    else {
    createEditor();
    }
    });

    function createEditor() {
    StackExchange.prepareEditor({
    heartbeatType: 'answer',
    autoActivateHeartbeat: false,
    convertImagesToLinks: true,
    noModals: true,
    showLowRepImageUploadWarning: true,
    reputationToPostImages: 10,
    bindNavPrevention: true,
    postfix: "",
    imageUploader: {
    brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
    contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
    allowUrls: true
    },
    onDemand: true,
    discardSelector: ".discard-answer"
    ,immediatelyShowMarkdownHelp:true
    });


    }
    });














    draft saved

    draft discarded


















    StackExchange.ready(
    function () {
    StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53334276%2fresample-or-aggregate-large-raster-limiting-memory-use-r%23new-answer', 'question_page');
    }
    );

    Post as a guest















    Required, but never shown

























    1 Answer
    1






    active

    oldest

    votes








    1 Answer
    1






    active

    oldest

    votes









    active

    oldest

    votes






    active

    oldest

    votes









    0














    You probably should update the raster package. The previous short-lived release (2.7-15) had an error in the memory settings. Version 2.8-4 should have fixed that.






    share|improve this answer




























      0














      You probably should update the raster package. The previous short-lived release (2.7-15) had an error in the memory settings. Version 2.8-4 should have fixed that.






      share|improve this answer


























        0












        0








        0







        You probably should update the raster package. The previous short-lived release (2.7-15) had an error in the memory settings. Version 2.8-4 should have fixed that.






        share|improve this answer













        You probably should update the raster package. The previous short-lived release (2.7-15) had an error in the memory settings. Version 2.8-4 should have fixed that.







        share|improve this answer












        share|improve this answer



        share|improve this answer










        answered Nov 17 '18 at 4:19









        Robert HijmansRobert Hijmans

        13.1k12428




        13.1k12428






























            draft saved

            draft discarded




















































            Thanks for contributing an answer to Stack Overflow!


            • Please be sure to answer the question. Provide details and share your research!

            But avoid



            • Asking for help, clarification, or responding to other answers.

            • Making statements based on opinion; back them up with references or personal experience.


            To learn more, see our tips on writing great answers.




            draft saved


            draft discarded














            StackExchange.ready(
            function () {
            StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53334276%2fresample-or-aggregate-large-raster-limiting-memory-use-r%23new-answer', 'question_page');
            }
            );

            Post as a guest















            Required, but never shown





















































            Required, but never shown














            Required, but never shown












            Required, but never shown







            Required, but never shown

































            Required, but never shown














            Required, but never shown












            Required, but never shown







            Required, but never shown







            Popular posts from this blog

            Guess what letter conforming each word

            Port of Spain

            Run scheduled task as local user group (not BUILTIN)