Reorganizing a 3d numpy array





.everyoneloves__top-leaderboard:empty,.everyoneloves__mid-leaderboard:empty,.everyoneloves__bot-mid-leaderboard:empty{ height:90px;width:728px;box-sizing:border-box;
}







0















I've tried and searched for a few days, I've come closer but need your help.



I have a 3d array in python,



shape(files)
>> (31,2049,2)


which corresponds to 31 input files with 2 columns of data with 2048 rows and a header.



I'd like to sort this array based on the header, which is a number, in each file.



I tried to follow NumPy: sorting 3D array but keeping 2nd dimension assigned to first , but i'm incredibly confused.



First I try to setup get my headers for the argsort, I thought I could do



sortval=files[:][0][0]


but this does not work..



Then I simply did a for loop to iterate and get my headers



for i in xrange(shape(files)[0]:
sortval.append([i][0][0])


Then



sortedIdx = np.argsort(sortval)


This works, however I dont understand whats happening in the last line..



files = files[np.arange(len(deck))[:,np.newaxis],sortedIdx]


Help would be appreciated.










share|improve this question































    0















    I've tried and searched for a few days, I've come closer but need your help.



    I have a 3d array in python,



    shape(files)
    >> (31,2049,2)


    which corresponds to 31 input files with 2 columns of data with 2048 rows and a header.



    I'd like to sort this array based on the header, which is a number, in each file.



    I tried to follow NumPy: sorting 3D array but keeping 2nd dimension assigned to first , but i'm incredibly confused.



    First I try to setup get my headers for the argsort, I thought I could do



    sortval=files[:][0][0]


    but this does not work..



    Then I simply did a for loop to iterate and get my headers



    for i in xrange(shape(files)[0]:
    sortval.append([i][0][0])


    Then



    sortedIdx = np.argsort(sortval)


    This works, however I dont understand whats happening in the last line..



    files = files[np.arange(len(deck))[:,np.newaxis],sortedIdx]


    Help would be appreciated.










    share|improve this question



























      0












      0








      0








      I've tried and searched for a few days, I've come closer but need your help.



      I have a 3d array in python,



      shape(files)
      >> (31,2049,2)


      which corresponds to 31 input files with 2 columns of data with 2048 rows and a header.



      I'd like to sort this array based on the header, which is a number, in each file.



      I tried to follow NumPy: sorting 3D array but keeping 2nd dimension assigned to first , but i'm incredibly confused.



      First I try to setup get my headers for the argsort, I thought I could do



      sortval=files[:][0][0]


      but this does not work..



      Then I simply did a for loop to iterate and get my headers



      for i in xrange(shape(files)[0]:
      sortval.append([i][0][0])


      Then



      sortedIdx = np.argsort(sortval)


      This works, however I dont understand whats happening in the last line..



      files = files[np.arange(len(deck))[:,np.newaxis],sortedIdx]


      Help would be appreciated.










      share|improve this question
















      I've tried and searched for a few days, I've come closer but need your help.



      I have a 3d array in python,



      shape(files)
      >> (31,2049,2)


      which corresponds to 31 input files with 2 columns of data with 2048 rows and a header.



      I'd like to sort this array based on the header, which is a number, in each file.



      I tried to follow NumPy: sorting 3D array but keeping 2nd dimension assigned to first , but i'm incredibly confused.



      First I try to setup get my headers for the argsort, I thought I could do



      sortval=files[:][0][0]


      but this does not work..



      Then I simply did a for loop to iterate and get my headers



      for i in xrange(shape(files)[0]:
      sortval.append([i][0][0])


      Then



      sortedIdx = np.argsort(sortval)


      This works, however I dont understand whats happening in the last line..



      files = files[np.arange(len(deck))[:,np.newaxis],sortedIdx]


      Help would be appreciated.







      python numpy matplotlib






      share|improve this question















      share|improve this question













      share|improve this question




      share|improve this question








      edited Nov 21 '18 at 23:18









      ImportanceOfBeingErnest

      141k13166243




      141k13166243










      asked Nov 21 '18 at 22:59









      LearningPythonLearningPython

      32




      32
























          2 Answers
          2






          active

          oldest

          votes


















          0














          Here we can use a simple example to demonstrate what your code is doing:



          First we create a random 3D numpy matrix:



          a = (np.random.rand(3,3,2)*10).astype(int)
          array([[[3, 1],
          [3, 7],
          [0, 3]],
          [[2, 9],
          [1, 0],
          [9, 2]],
          [[9, 2],
          [8, 8],
          [8, 0]]])


          Then a[:] will gives a itself, and a[:][0][0] is just the first row in first 2D array in a, which is:



          a[:][0]
          # array([[3, 1],
          # [3, 7],
          # [0, 3]])

          a[:][0][0]
          # array([3, 1])


          What you want is the header which are 3,2,9 in this example, so we can use a[:, 0, 0] to extract them:



          a[:,0,0]
          # array([3, 2, 9])


          Now we sort the above list and get an index array:



          np.argsort(a[:,0,0])
          # array([1, 0, 2])


          In order to rearrange the entire 3D array, we need to slice the array with correct order. And np.arange(len(a))[:,np.newaxis] is equal to np.arange(len(a)).reshape(-1,1) which creates a sequential 2D index array:



          np.arange(len(a))[:,np.newaxis]
          # array([[0],
          # [1],
          # [2]])


          Without the 2D array, we will slice the array to 2 dimension



          a[np.arange(3), np.argsort(a[:,0,0])]
          # array([[3, 7],
          # [2, 9],
          # [8, 0]])


          With the 2D array, we can perform 3D slicing and keeps the shape:



          a[np.arange(3).reshape(-1,1), np.argsort(a[:,0,0])]
          array([[[3, 7],
          [3, 1],
          [0, 3]],
          [[1, 0],
          [2, 9],
          [9, 2]],
          [[8, 8],
          [9, 2],
          [8, 0]]])


          And above is the final result you want.



          Edit:



          To arange the 2D arrays:, one could use:



          a[np.argsort(a[:,0,0])]
          array([[[2, 9],
          [1, 0],
          [9, 2]],
          [[3, 1],
          [3, 7],
          [0, 3]],
          [[9, 2],
          [8, 8],
          [8, 0]]])





          share|improve this answer


























          • Dear Kevin, thank you for the reply. However I do not want to change the individual sub-arrays, only rearrange them within the 3D array. e.g. in your example, to finish with [[[2,9],,],[3,1],,],[9,2],,]. I can reword the question to clarify.

            – LearningPython
            Nov 21 '18 at 23:51











          • @LearningPython then a[np.argsort(a[:,0,0])] would do that.

            – Kevin Fang
            Nov 21 '18 at 23:58



















          1














          Another way to do this is with np.take



          header = a[:,0,0]
          sorted = np.take(a, np.argsort(header), axis=0)





          share|improve this answer
























            Your Answer






            StackExchange.ifUsing("editor", function () {
            StackExchange.using("externalEditor", function () {
            StackExchange.using("snippets", function () {
            StackExchange.snippets.init();
            });
            });
            }, "code-snippets");

            StackExchange.ready(function() {
            var channelOptions = {
            tags: "".split(" "),
            id: "1"
            };
            initTagRenderer("".split(" "), "".split(" "), channelOptions);

            StackExchange.using("externalEditor", function() {
            // Have to fire editor after snippets, if snippets enabled
            if (StackExchange.settings.snippets.snippetsEnabled) {
            StackExchange.using("snippets", function() {
            createEditor();
            });
            }
            else {
            createEditor();
            }
            });

            function createEditor() {
            StackExchange.prepareEditor({
            heartbeatType: 'answer',
            autoActivateHeartbeat: false,
            convertImagesToLinks: true,
            noModals: true,
            showLowRepImageUploadWarning: true,
            reputationToPostImages: 10,
            bindNavPrevention: true,
            postfix: "",
            imageUploader: {
            brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
            contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
            allowUrls: true
            },
            onDemand: true,
            discardSelector: ".discard-answer"
            ,immediatelyShowMarkdownHelp:true
            });


            }
            });














            draft saved

            draft discarded


















            StackExchange.ready(
            function () {
            StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53421637%2freorganizing-a-3d-numpy-array%23new-answer', 'question_page');
            }
            );

            Post as a guest















            Required, but never shown

























            2 Answers
            2






            active

            oldest

            votes








            2 Answers
            2






            active

            oldest

            votes









            active

            oldest

            votes






            active

            oldest

            votes









            0














            Here we can use a simple example to demonstrate what your code is doing:



            First we create a random 3D numpy matrix:



            a = (np.random.rand(3,3,2)*10).astype(int)
            array([[[3, 1],
            [3, 7],
            [0, 3]],
            [[2, 9],
            [1, 0],
            [9, 2]],
            [[9, 2],
            [8, 8],
            [8, 0]]])


            Then a[:] will gives a itself, and a[:][0][0] is just the first row in first 2D array in a, which is:



            a[:][0]
            # array([[3, 1],
            # [3, 7],
            # [0, 3]])

            a[:][0][0]
            # array([3, 1])


            What you want is the header which are 3,2,9 in this example, so we can use a[:, 0, 0] to extract them:



            a[:,0,0]
            # array([3, 2, 9])


            Now we sort the above list and get an index array:



            np.argsort(a[:,0,0])
            # array([1, 0, 2])


            In order to rearrange the entire 3D array, we need to slice the array with correct order. And np.arange(len(a))[:,np.newaxis] is equal to np.arange(len(a)).reshape(-1,1) which creates a sequential 2D index array:



            np.arange(len(a))[:,np.newaxis]
            # array([[0],
            # [1],
            # [2]])


            Without the 2D array, we will slice the array to 2 dimension



            a[np.arange(3), np.argsort(a[:,0,0])]
            # array([[3, 7],
            # [2, 9],
            # [8, 0]])


            With the 2D array, we can perform 3D slicing and keeps the shape:



            a[np.arange(3).reshape(-1,1), np.argsort(a[:,0,0])]
            array([[[3, 7],
            [3, 1],
            [0, 3]],
            [[1, 0],
            [2, 9],
            [9, 2]],
            [[8, 8],
            [9, 2],
            [8, 0]]])


            And above is the final result you want.



            Edit:



            To arange the 2D arrays:, one could use:



            a[np.argsort(a[:,0,0])]
            array([[[2, 9],
            [1, 0],
            [9, 2]],
            [[3, 1],
            [3, 7],
            [0, 3]],
            [[9, 2],
            [8, 8],
            [8, 0]]])





            share|improve this answer


























            • Dear Kevin, thank you for the reply. However I do not want to change the individual sub-arrays, only rearrange them within the 3D array. e.g. in your example, to finish with [[[2,9],,],[3,1],,],[9,2],,]. I can reword the question to clarify.

              – LearningPython
              Nov 21 '18 at 23:51











            • @LearningPython then a[np.argsort(a[:,0,0])] would do that.

              – Kevin Fang
              Nov 21 '18 at 23:58
















            0














            Here we can use a simple example to demonstrate what your code is doing:



            First we create a random 3D numpy matrix:



            a = (np.random.rand(3,3,2)*10).astype(int)
            array([[[3, 1],
            [3, 7],
            [0, 3]],
            [[2, 9],
            [1, 0],
            [9, 2]],
            [[9, 2],
            [8, 8],
            [8, 0]]])


            Then a[:] will gives a itself, and a[:][0][0] is just the first row in first 2D array in a, which is:



            a[:][0]
            # array([[3, 1],
            # [3, 7],
            # [0, 3]])

            a[:][0][0]
            # array([3, 1])


            What you want is the header which are 3,2,9 in this example, so we can use a[:, 0, 0] to extract them:



            a[:,0,0]
            # array([3, 2, 9])


            Now we sort the above list and get an index array:



            np.argsort(a[:,0,0])
            # array([1, 0, 2])


            In order to rearrange the entire 3D array, we need to slice the array with correct order. And np.arange(len(a))[:,np.newaxis] is equal to np.arange(len(a)).reshape(-1,1) which creates a sequential 2D index array:



            np.arange(len(a))[:,np.newaxis]
            # array([[0],
            # [1],
            # [2]])


            Without the 2D array, we will slice the array to 2 dimension



            a[np.arange(3), np.argsort(a[:,0,0])]
            # array([[3, 7],
            # [2, 9],
            # [8, 0]])


            With the 2D array, we can perform 3D slicing and keeps the shape:



            a[np.arange(3).reshape(-1,1), np.argsort(a[:,0,0])]
            array([[[3, 7],
            [3, 1],
            [0, 3]],
            [[1, 0],
            [2, 9],
            [9, 2]],
            [[8, 8],
            [9, 2],
            [8, 0]]])


            And above is the final result you want.



            Edit:



            To arange the 2D arrays:, one could use:



            a[np.argsort(a[:,0,0])]
            array([[[2, 9],
            [1, 0],
            [9, 2]],
            [[3, 1],
            [3, 7],
            [0, 3]],
            [[9, 2],
            [8, 8],
            [8, 0]]])





            share|improve this answer


























            • Dear Kevin, thank you for the reply. However I do not want to change the individual sub-arrays, only rearrange them within the 3D array. e.g. in your example, to finish with [[[2,9],,],[3,1],,],[9,2],,]. I can reword the question to clarify.

              – LearningPython
              Nov 21 '18 at 23:51











            • @LearningPython then a[np.argsort(a[:,0,0])] would do that.

              – Kevin Fang
              Nov 21 '18 at 23:58














            0












            0








            0







            Here we can use a simple example to demonstrate what your code is doing:



            First we create a random 3D numpy matrix:



            a = (np.random.rand(3,3,2)*10).astype(int)
            array([[[3, 1],
            [3, 7],
            [0, 3]],
            [[2, 9],
            [1, 0],
            [9, 2]],
            [[9, 2],
            [8, 8],
            [8, 0]]])


            Then a[:] will gives a itself, and a[:][0][0] is just the first row in first 2D array in a, which is:



            a[:][0]
            # array([[3, 1],
            # [3, 7],
            # [0, 3]])

            a[:][0][0]
            # array([3, 1])


            What you want is the header which are 3,2,9 in this example, so we can use a[:, 0, 0] to extract them:



            a[:,0,0]
            # array([3, 2, 9])


            Now we sort the above list and get an index array:



            np.argsort(a[:,0,0])
            # array([1, 0, 2])


            In order to rearrange the entire 3D array, we need to slice the array with correct order. And np.arange(len(a))[:,np.newaxis] is equal to np.arange(len(a)).reshape(-1,1) which creates a sequential 2D index array:



            np.arange(len(a))[:,np.newaxis]
            # array([[0],
            # [1],
            # [2]])


            Without the 2D array, we will slice the array to 2 dimension



            a[np.arange(3), np.argsort(a[:,0,0])]
            # array([[3, 7],
            # [2, 9],
            # [8, 0]])


            With the 2D array, we can perform 3D slicing and keeps the shape:



            a[np.arange(3).reshape(-1,1), np.argsort(a[:,0,0])]
            array([[[3, 7],
            [3, 1],
            [0, 3]],
            [[1, 0],
            [2, 9],
            [9, 2]],
            [[8, 8],
            [9, 2],
            [8, 0]]])


            And above is the final result you want.



            Edit:



            To arange the 2D arrays:, one could use:



            a[np.argsort(a[:,0,0])]
            array([[[2, 9],
            [1, 0],
            [9, 2]],
            [[3, 1],
            [3, 7],
            [0, 3]],
            [[9, 2],
            [8, 8],
            [8, 0]]])





            share|improve this answer















            Here we can use a simple example to demonstrate what your code is doing:



            First we create a random 3D numpy matrix:



            a = (np.random.rand(3,3,2)*10).astype(int)
            array([[[3, 1],
            [3, 7],
            [0, 3]],
            [[2, 9],
            [1, 0],
            [9, 2]],
            [[9, 2],
            [8, 8],
            [8, 0]]])


            Then a[:] will gives a itself, and a[:][0][0] is just the first row in first 2D array in a, which is:



            a[:][0]
            # array([[3, 1],
            # [3, 7],
            # [0, 3]])

            a[:][0][0]
            # array([3, 1])


            What you want is the header which are 3,2,9 in this example, so we can use a[:, 0, 0] to extract them:



            a[:,0,0]
            # array([3, 2, 9])


            Now we sort the above list and get an index array:



            np.argsort(a[:,0,0])
            # array([1, 0, 2])


            In order to rearrange the entire 3D array, we need to slice the array with correct order. And np.arange(len(a))[:,np.newaxis] is equal to np.arange(len(a)).reshape(-1,1) which creates a sequential 2D index array:



            np.arange(len(a))[:,np.newaxis]
            # array([[0],
            # [1],
            # [2]])


            Without the 2D array, we will slice the array to 2 dimension



            a[np.arange(3), np.argsort(a[:,0,0])]
            # array([[3, 7],
            # [2, 9],
            # [8, 0]])


            With the 2D array, we can perform 3D slicing and keeps the shape:



            a[np.arange(3).reshape(-1,1), np.argsort(a[:,0,0])]
            array([[[3, 7],
            [3, 1],
            [0, 3]],
            [[1, 0],
            [2, 9],
            [9, 2]],
            [[8, 8],
            [9, 2],
            [8, 0]]])


            And above is the final result you want.



            Edit:



            To arange the 2D arrays:, one could use:



            a[np.argsort(a[:,0,0])]
            array([[[2, 9],
            [1, 0],
            [9, 2]],
            [[3, 1],
            [3, 7],
            [0, 3]],
            [[9, 2],
            [8, 8],
            [8, 0]]])






            share|improve this answer














            share|improve this answer



            share|improve this answer








            edited Nov 21 '18 at 23:59

























            answered Nov 21 '18 at 23:30









            Kevin FangKevin Fang

            1,296318




            1,296318













            • Dear Kevin, thank you for the reply. However I do not want to change the individual sub-arrays, only rearrange them within the 3D array. e.g. in your example, to finish with [[[2,9],,],[3,1],,],[9,2],,]. I can reword the question to clarify.

              – LearningPython
              Nov 21 '18 at 23:51











            • @LearningPython then a[np.argsort(a[:,0,0])] would do that.

              – Kevin Fang
              Nov 21 '18 at 23:58



















            • Dear Kevin, thank you for the reply. However I do not want to change the individual sub-arrays, only rearrange them within the 3D array. e.g. in your example, to finish with [[[2,9],,],[3,1],,],[9,2],,]. I can reword the question to clarify.

              – LearningPython
              Nov 21 '18 at 23:51











            • @LearningPython then a[np.argsort(a[:,0,0])] would do that.

              – Kevin Fang
              Nov 21 '18 at 23:58

















            Dear Kevin, thank you for the reply. However I do not want to change the individual sub-arrays, only rearrange them within the 3D array. e.g. in your example, to finish with [[[2,9],,],[3,1],,],[9,2],,]. I can reword the question to clarify.

            – LearningPython
            Nov 21 '18 at 23:51





            Dear Kevin, thank you for the reply. However I do not want to change the individual sub-arrays, only rearrange them within the 3D array. e.g. in your example, to finish with [[[2,9],,],[3,1],,],[9,2],,]. I can reword the question to clarify.

            – LearningPython
            Nov 21 '18 at 23:51













            @LearningPython then a[np.argsort(a[:,0,0])] would do that.

            – Kevin Fang
            Nov 21 '18 at 23:58





            @LearningPython then a[np.argsort(a[:,0,0])] would do that.

            – Kevin Fang
            Nov 21 '18 at 23:58













            1














            Another way to do this is with np.take



            header = a[:,0,0]
            sorted = np.take(a, np.argsort(header), axis=0)





            share|improve this answer




























              1














              Another way to do this is with np.take



              header = a[:,0,0]
              sorted = np.take(a, np.argsort(header), axis=0)





              share|improve this answer


























                1












                1








                1







                Another way to do this is with np.take



                header = a[:,0,0]
                sorted = np.take(a, np.argsort(header), axis=0)





                share|improve this answer













                Another way to do this is with np.take



                header = a[:,0,0]
                sorted = np.take(a, np.argsort(header), axis=0)






                share|improve this answer












                share|improve this answer



                share|improve this answer










                answered Nov 22 '18 at 3:16









                EricEric

                67.5k33170284




                67.5k33170284






























                    draft saved

                    draft discarded




















































                    Thanks for contributing an answer to Stack Overflow!


                    • Please be sure to answer the question. Provide details and share your research!

                    But avoid



                    • Asking for help, clarification, or responding to other answers.

                    • Making statements based on opinion; back them up with references or personal experience.


                    To learn more, see our tips on writing great answers.




                    draft saved


                    draft discarded














                    StackExchange.ready(
                    function () {
                    StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53421637%2freorganizing-a-3d-numpy-array%23new-answer', 'question_page');
                    }
                    );

                    Post as a guest















                    Required, but never shown





















































                    Required, but never shown














                    Required, but never shown












                    Required, but never shown







                    Required, but never shown

































                    Required, but never shown














                    Required, but never shown












                    Required, but never shown







                    Required, but never shown







                    Popular posts from this blog

                    Guess what letter conforming each word

                    Port of Spain

                    Run scheduled task as local user group (not BUILTIN)