With Python ruamel.yaml, lost anchor when loading in round-trip mode?












1















When loading a file containing anchors at different nesting levels, the anchor seems lost and the key remains empty.



When loading the following file:



---
Dict1:
- InnerDict: &inner
key: val

Dict2:
InnerDict:
<<: *inner

Dict3:
- InnerDict:
<<: *inner
...


... with the code (Python 3.7, ruamel.yaml version 0.15.78):



from ruamel.yaml import YAML
with open("file.yaml") as infile:
content = YAML(typ='rt', pure=True).load(infile)
print(content)


... gives:



{'Dict1': [ordereddict([('InnerDict', ordereddict([('key', 'val')]))])],
'Dict2': ordereddict([('InnerDict', ordereddict())]),
'Dict3': [ordereddict([('InnerDict', ordereddict([('key', 'val')]))])]}


... where the inner dict of Dict2 remains empty.



In safe mode, the anchor is interpreted as expected in both cases.



Is it intended or is it a bug ?










share|improve this question



























    1















    When loading a file containing anchors at different nesting levels, the anchor seems lost and the key remains empty.



    When loading the following file:



    ---
    Dict1:
    - InnerDict: &inner
    key: val

    Dict2:
    InnerDict:
    <<: *inner

    Dict3:
    - InnerDict:
    <<: *inner
    ...


    ... with the code (Python 3.7, ruamel.yaml version 0.15.78):



    from ruamel.yaml import YAML
    with open("file.yaml") as infile:
    content = YAML(typ='rt', pure=True).load(infile)
    print(content)


    ... gives:



    {'Dict1': [ordereddict([('InnerDict', ordereddict([('key', 'val')]))])],
    'Dict2': ordereddict([('InnerDict', ordereddict())]),
    'Dict3': [ordereddict([('InnerDict', ordereddict([('key', 'val')]))])]}


    ... where the inner dict of Dict2 remains empty.



    In safe mode, the anchor is interpreted as expected in both cases.



    Is it intended or is it a bug ?










    share|improve this question

























      1












      1








      1








      When loading a file containing anchors at different nesting levels, the anchor seems lost and the key remains empty.



      When loading the following file:



      ---
      Dict1:
      - InnerDict: &inner
      key: val

      Dict2:
      InnerDict:
      <<: *inner

      Dict3:
      - InnerDict:
      <<: *inner
      ...


      ... with the code (Python 3.7, ruamel.yaml version 0.15.78):



      from ruamel.yaml import YAML
      with open("file.yaml") as infile:
      content = YAML(typ='rt', pure=True).load(infile)
      print(content)


      ... gives:



      {'Dict1': [ordereddict([('InnerDict', ordereddict([('key', 'val')]))])],
      'Dict2': ordereddict([('InnerDict', ordereddict())]),
      'Dict3': [ordereddict([('InnerDict', ordereddict([('key', 'val')]))])]}


      ... where the inner dict of Dict2 remains empty.



      In safe mode, the anchor is interpreted as expected in both cases.



      Is it intended or is it a bug ?










      share|improve this question














      When loading a file containing anchors at different nesting levels, the anchor seems lost and the key remains empty.



      When loading the following file:



      ---
      Dict1:
      - InnerDict: &inner
      key: val

      Dict2:
      InnerDict:
      <<: *inner

      Dict3:
      - InnerDict:
      <<: *inner
      ...


      ... with the code (Python 3.7, ruamel.yaml version 0.15.78):



      from ruamel.yaml import YAML
      with open("file.yaml") as infile:
      content = YAML(typ='rt', pure=True).load(infile)
      print(content)


      ... gives:



      {'Dict1': [ordereddict([('InnerDict', ordereddict([('key', 'val')]))])],
      'Dict2': ordereddict([('InnerDict', ordereddict())]),
      'Dict3': [ordereddict([('InnerDict', ordereddict([('key', 'val')]))])]}


      ... where the inner dict of Dict2 remains empty.



      In safe mode, the anchor is interpreted as expected in both cases.



      Is it intended or is it a bug ?







      python ruamel.yaml






      share|improve this question













      share|improve this question











      share|improve this question




      share|improve this question










      asked Nov 20 '18 at 15:56









      gaFFgaFF

      646




      646
























          1 Answer
          1






          active

          oldest

          votes


















          0














          That is definitely a bug, and it has to do with the breadth-first
          construction of mappings in the data structure. By the time the
          InnerDict under Dict2 is constructed, the one nested under Dict1
          is not completely available. The InnerDict under Dict3 lies at the
          same depth, and can therefore correctly be constructed (and so can the
          first merge if you remove the dash in the Dict1 construction and so
          making the anchor appear at a shallower place).



          One way to solve this, apart from installing ruamel.yaml>=0.15.79, is by
          providing an alternative constructor, that forces depth first processing:



          import sys
          import ruamel.yaml

          class MyConstructor(ruamel.yaml.constructor.RoundTripConstructor):
          def construct_yaml_map(self, node):
          data = ruamel.yaml.comments.CommentedMap()
          data._yaml_set_line_col(node.start_mark.line, node.start_mark.column)
          yield data
          self.construct_mapping(node, data, deep=True)
          self.set_collection_style(data, node)

          MyConstructor.add_constructor(
          u'tag:yaml.org,2002:map', MyConstructor.construct_yaml_map
          )


          yaml = ruamel.yaml.YAML()
          yaml.Constructor = MyConstructor


          yaml_str = """
          Dict1:
          - InnerDict: &inner
          key: val

          Dict2:
          InnerDict:
          <<: *inner

          Dict3:
          - InnerDict:
          <<: *inner
          """

          data = yaml.load(yaml_str)
          for k in data:
          print(k, data[k])
          print('---------')
          yaml.dump(data, sys.stdout)


          Generating:



          Dict1 [ordereddict([('InnerDict', ordereddict([('key', 'val')]))])]
          Dict2 ordereddict([('InnerDict', ordereddict([('key', 'val')]))])
          Dict3 [ordereddict([('InnerDict', ordereddict([('key', 'val')]))])]
          ---------
          Dict1:
          - InnerDict: &inner
          key: val

          Dict2:
          InnerDict:
          <<: *inner

          Dict3:
          - InnerDict:
          <<: *inner


          (As the dump of data is correct even without the above "patch", this
          was not previously detected when testing round-trips)






          share|improve this answer

























            Your Answer






            StackExchange.ifUsing("editor", function () {
            StackExchange.using("externalEditor", function () {
            StackExchange.using("snippets", function () {
            StackExchange.snippets.init();
            });
            });
            }, "code-snippets");

            StackExchange.ready(function() {
            var channelOptions = {
            tags: "".split(" "),
            id: "1"
            };
            initTagRenderer("".split(" "), "".split(" "), channelOptions);

            StackExchange.using("externalEditor", function() {
            // Have to fire editor after snippets, if snippets enabled
            if (StackExchange.settings.snippets.snippetsEnabled) {
            StackExchange.using("snippets", function() {
            createEditor();
            });
            }
            else {
            createEditor();
            }
            });

            function createEditor() {
            StackExchange.prepareEditor({
            heartbeatType: 'answer',
            autoActivateHeartbeat: false,
            convertImagesToLinks: true,
            noModals: true,
            showLowRepImageUploadWarning: true,
            reputationToPostImages: 10,
            bindNavPrevention: true,
            postfix: "",
            imageUploader: {
            brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
            contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
            allowUrls: true
            },
            onDemand: true,
            discardSelector: ".discard-answer"
            ,immediatelyShowMarkdownHelp:true
            });


            }
            });














            draft saved

            draft discarded


















            StackExchange.ready(
            function () {
            StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53396845%2fwith-python-ruamel-yaml-lost-anchor-when-loading-in-round-trip-mode%23new-answer', 'question_page');
            }
            );

            Post as a guest















            Required, but never shown

























            1 Answer
            1






            active

            oldest

            votes








            1 Answer
            1






            active

            oldest

            votes









            active

            oldest

            votes






            active

            oldest

            votes









            0














            That is definitely a bug, and it has to do with the breadth-first
            construction of mappings in the data structure. By the time the
            InnerDict under Dict2 is constructed, the one nested under Dict1
            is not completely available. The InnerDict under Dict3 lies at the
            same depth, and can therefore correctly be constructed (and so can the
            first merge if you remove the dash in the Dict1 construction and so
            making the anchor appear at a shallower place).



            One way to solve this, apart from installing ruamel.yaml>=0.15.79, is by
            providing an alternative constructor, that forces depth first processing:



            import sys
            import ruamel.yaml

            class MyConstructor(ruamel.yaml.constructor.RoundTripConstructor):
            def construct_yaml_map(self, node):
            data = ruamel.yaml.comments.CommentedMap()
            data._yaml_set_line_col(node.start_mark.line, node.start_mark.column)
            yield data
            self.construct_mapping(node, data, deep=True)
            self.set_collection_style(data, node)

            MyConstructor.add_constructor(
            u'tag:yaml.org,2002:map', MyConstructor.construct_yaml_map
            )


            yaml = ruamel.yaml.YAML()
            yaml.Constructor = MyConstructor


            yaml_str = """
            Dict1:
            - InnerDict: &inner
            key: val

            Dict2:
            InnerDict:
            <<: *inner

            Dict3:
            - InnerDict:
            <<: *inner
            """

            data = yaml.load(yaml_str)
            for k in data:
            print(k, data[k])
            print('---------')
            yaml.dump(data, sys.stdout)


            Generating:



            Dict1 [ordereddict([('InnerDict', ordereddict([('key', 'val')]))])]
            Dict2 ordereddict([('InnerDict', ordereddict([('key', 'val')]))])
            Dict3 [ordereddict([('InnerDict', ordereddict([('key', 'val')]))])]
            ---------
            Dict1:
            - InnerDict: &inner
            key: val

            Dict2:
            InnerDict:
            <<: *inner

            Dict3:
            - InnerDict:
            <<: *inner


            (As the dump of data is correct even without the above "patch", this
            was not previously detected when testing round-trips)






            share|improve this answer






























              0














              That is definitely a bug, and it has to do with the breadth-first
              construction of mappings in the data structure. By the time the
              InnerDict under Dict2 is constructed, the one nested under Dict1
              is not completely available. The InnerDict under Dict3 lies at the
              same depth, and can therefore correctly be constructed (and so can the
              first merge if you remove the dash in the Dict1 construction and so
              making the anchor appear at a shallower place).



              One way to solve this, apart from installing ruamel.yaml>=0.15.79, is by
              providing an alternative constructor, that forces depth first processing:



              import sys
              import ruamel.yaml

              class MyConstructor(ruamel.yaml.constructor.RoundTripConstructor):
              def construct_yaml_map(self, node):
              data = ruamel.yaml.comments.CommentedMap()
              data._yaml_set_line_col(node.start_mark.line, node.start_mark.column)
              yield data
              self.construct_mapping(node, data, deep=True)
              self.set_collection_style(data, node)

              MyConstructor.add_constructor(
              u'tag:yaml.org,2002:map', MyConstructor.construct_yaml_map
              )


              yaml = ruamel.yaml.YAML()
              yaml.Constructor = MyConstructor


              yaml_str = """
              Dict1:
              - InnerDict: &inner
              key: val

              Dict2:
              InnerDict:
              <<: *inner

              Dict3:
              - InnerDict:
              <<: *inner
              """

              data = yaml.load(yaml_str)
              for k in data:
              print(k, data[k])
              print('---------')
              yaml.dump(data, sys.stdout)


              Generating:



              Dict1 [ordereddict([('InnerDict', ordereddict([('key', 'val')]))])]
              Dict2 ordereddict([('InnerDict', ordereddict([('key', 'val')]))])
              Dict3 [ordereddict([('InnerDict', ordereddict([('key', 'val')]))])]
              ---------
              Dict1:
              - InnerDict: &inner
              key: val

              Dict2:
              InnerDict:
              <<: *inner

              Dict3:
              - InnerDict:
              <<: *inner


              (As the dump of data is correct even without the above "patch", this
              was not previously detected when testing round-trips)






              share|improve this answer




























                0












                0








                0







                That is definitely a bug, and it has to do with the breadth-first
                construction of mappings in the data structure. By the time the
                InnerDict under Dict2 is constructed, the one nested under Dict1
                is not completely available. The InnerDict under Dict3 lies at the
                same depth, and can therefore correctly be constructed (and so can the
                first merge if you remove the dash in the Dict1 construction and so
                making the anchor appear at a shallower place).



                One way to solve this, apart from installing ruamel.yaml>=0.15.79, is by
                providing an alternative constructor, that forces depth first processing:



                import sys
                import ruamel.yaml

                class MyConstructor(ruamel.yaml.constructor.RoundTripConstructor):
                def construct_yaml_map(self, node):
                data = ruamel.yaml.comments.CommentedMap()
                data._yaml_set_line_col(node.start_mark.line, node.start_mark.column)
                yield data
                self.construct_mapping(node, data, deep=True)
                self.set_collection_style(data, node)

                MyConstructor.add_constructor(
                u'tag:yaml.org,2002:map', MyConstructor.construct_yaml_map
                )


                yaml = ruamel.yaml.YAML()
                yaml.Constructor = MyConstructor


                yaml_str = """
                Dict1:
                - InnerDict: &inner
                key: val

                Dict2:
                InnerDict:
                <<: *inner

                Dict3:
                - InnerDict:
                <<: *inner
                """

                data = yaml.load(yaml_str)
                for k in data:
                print(k, data[k])
                print('---------')
                yaml.dump(data, sys.stdout)


                Generating:



                Dict1 [ordereddict([('InnerDict', ordereddict([('key', 'val')]))])]
                Dict2 ordereddict([('InnerDict', ordereddict([('key', 'val')]))])
                Dict3 [ordereddict([('InnerDict', ordereddict([('key', 'val')]))])]
                ---------
                Dict1:
                - InnerDict: &inner
                key: val

                Dict2:
                InnerDict:
                <<: *inner

                Dict3:
                - InnerDict:
                <<: *inner


                (As the dump of data is correct even without the above "patch", this
                was not previously detected when testing round-trips)






                share|improve this answer















                That is definitely a bug, and it has to do with the breadth-first
                construction of mappings in the data structure. By the time the
                InnerDict under Dict2 is constructed, the one nested under Dict1
                is not completely available. The InnerDict under Dict3 lies at the
                same depth, and can therefore correctly be constructed (and so can the
                first merge if you remove the dash in the Dict1 construction and so
                making the anchor appear at a shallower place).



                One way to solve this, apart from installing ruamel.yaml>=0.15.79, is by
                providing an alternative constructor, that forces depth first processing:



                import sys
                import ruamel.yaml

                class MyConstructor(ruamel.yaml.constructor.RoundTripConstructor):
                def construct_yaml_map(self, node):
                data = ruamel.yaml.comments.CommentedMap()
                data._yaml_set_line_col(node.start_mark.line, node.start_mark.column)
                yield data
                self.construct_mapping(node, data, deep=True)
                self.set_collection_style(data, node)

                MyConstructor.add_constructor(
                u'tag:yaml.org,2002:map', MyConstructor.construct_yaml_map
                )


                yaml = ruamel.yaml.YAML()
                yaml.Constructor = MyConstructor


                yaml_str = """
                Dict1:
                - InnerDict: &inner
                key: val

                Dict2:
                InnerDict:
                <<: *inner

                Dict3:
                - InnerDict:
                <<: *inner
                """

                data = yaml.load(yaml_str)
                for k in data:
                print(k, data[k])
                print('---------')
                yaml.dump(data, sys.stdout)


                Generating:



                Dict1 [ordereddict([('InnerDict', ordereddict([('key', 'val')]))])]
                Dict2 ordereddict([('InnerDict', ordereddict([('key', 'val')]))])
                Dict3 [ordereddict([('InnerDict', ordereddict([('key', 'val')]))])]
                ---------
                Dict1:
                - InnerDict: &inner
                key: val

                Dict2:
                InnerDict:
                <<: *inner

                Dict3:
                - InnerDict:
                <<: *inner


                (As the dump of data is correct even without the above "patch", this
                was not previously detected when testing round-trips)







                share|improve this answer














                share|improve this answer



                share|improve this answer








                edited Nov 21 '18 at 7:45

























                answered Nov 20 '18 at 16:50









                AnthonAnthon

                30.7k1795148




                30.7k1795148
































                    draft saved

                    draft discarded




















































                    Thanks for contributing an answer to Stack Overflow!


                    • Please be sure to answer the question. Provide details and share your research!

                    But avoid



                    • Asking for help, clarification, or responding to other answers.

                    • Making statements based on opinion; back them up with references or personal experience.


                    To learn more, see our tips on writing great answers.




                    draft saved


                    draft discarded














                    StackExchange.ready(
                    function () {
                    StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53396845%2fwith-python-ruamel-yaml-lost-anchor-when-loading-in-round-trip-mode%23new-answer', 'question_page');
                    }
                    );

                    Post as a guest















                    Required, but never shown





















































                    Required, but never shown














                    Required, but never shown












                    Required, but never shown







                    Required, but never shown

































                    Required, but never shown














                    Required, but never shown












                    Required, but never shown







                    Required, but never shown







                    Popular posts from this blog

                    Guess what letter conforming each word

                    Port of Spain

                    Run scheduled task as local user group (not BUILTIN)