With Python ruamel.yaml, lost anchor when loading in round-trip mode?
When loading a file containing anchors at different nesting levels, the anchor seems lost and the key remains empty.
When loading the following file:
---
Dict1:
- InnerDict: &inner
key: val
Dict2:
InnerDict:
<<: *inner
Dict3:
- InnerDict:
<<: *inner
...
... with the code (Python 3.7, ruamel.yaml version 0.15.78):
from ruamel.yaml import YAML
with open("file.yaml") as infile:
content = YAML(typ='rt', pure=True).load(infile)
print(content)
... gives:
{'Dict1': [ordereddict([('InnerDict', ordereddict([('key', 'val')]))])],
'Dict2': ordereddict([('InnerDict', ordereddict())]),
'Dict3': [ordereddict([('InnerDict', ordereddict([('key', 'val')]))])]}
... where the inner dict of Dict2
remains empty.
In safe mode, the anchor is interpreted as expected in both cases.
Is it intended or is it a bug ?
python ruamel.yaml
add a comment |
When loading a file containing anchors at different nesting levels, the anchor seems lost and the key remains empty.
When loading the following file:
---
Dict1:
- InnerDict: &inner
key: val
Dict2:
InnerDict:
<<: *inner
Dict3:
- InnerDict:
<<: *inner
...
... with the code (Python 3.7, ruamel.yaml version 0.15.78):
from ruamel.yaml import YAML
with open("file.yaml") as infile:
content = YAML(typ='rt', pure=True).load(infile)
print(content)
... gives:
{'Dict1': [ordereddict([('InnerDict', ordereddict([('key', 'val')]))])],
'Dict2': ordereddict([('InnerDict', ordereddict())]),
'Dict3': [ordereddict([('InnerDict', ordereddict([('key', 'val')]))])]}
... where the inner dict of Dict2
remains empty.
In safe mode, the anchor is interpreted as expected in both cases.
Is it intended or is it a bug ?
python ruamel.yaml
add a comment |
When loading a file containing anchors at different nesting levels, the anchor seems lost and the key remains empty.
When loading the following file:
---
Dict1:
- InnerDict: &inner
key: val
Dict2:
InnerDict:
<<: *inner
Dict3:
- InnerDict:
<<: *inner
...
... with the code (Python 3.7, ruamel.yaml version 0.15.78):
from ruamel.yaml import YAML
with open("file.yaml") as infile:
content = YAML(typ='rt', pure=True).load(infile)
print(content)
... gives:
{'Dict1': [ordereddict([('InnerDict', ordereddict([('key', 'val')]))])],
'Dict2': ordereddict([('InnerDict', ordereddict())]),
'Dict3': [ordereddict([('InnerDict', ordereddict([('key', 'val')]))])]}
... where the inner dict of Dict2
remains empty.
In safe mode, the anchor is interpreted as expected in both cases.
Is it intended or is it a bug ?
python ruamel.yaml
When loading a file containing anchors at different nesting levels, the anchor seems lost and the key remains empty.
When loading the following file:
---
Dict1:
- InnerDict: &inner
key: val
Dict2:
InnerDict:
<<: *inner
Dict3:
- InnerDict:
<<: *inner
...
... with the code (Python 3.7, ruamel.yaml version 0.15.78):
from ruamel.yaml import YAML
with open("file.yaml") as infile:
content = YAML(typ='rt', pure=True).load(infile)
print(content)
... gives:
{'Dict1': [ordereddict([('InnerDict', ordereddict([('key', 'val')]))])],
'Dict2': ordereddict([('InnerDict', ordereddict())]),
'Dict3': [ordereddict([('InnerDict', ordereddict([('key', 'val')]))])]}
... where the inner dict of Dict2
remains empty.
In safe mode, the anchor is interpreted as expected in both cases.
Is it intended or is it a bug ?
python ruamel.yaml
python ruamel.yaml
asked Nov 20 '18 at 15:56
gaFFgaFF
646
646
add a comment |
add a comment |
1 Answer
1
active
oldest
votes
That is definitely a bug, and it has to do with the breadth-first
construction of mappings in the data structure. By the time the
InnerDict
under Dict2
is constructed, the one nested under Dict1
is not completely available. The InnerDict
under Dict3
lies at the
same depth, and can therefore correctly be constructed (and so can the
first merge if you remove the dash in the Dict1
construction and so
making the anchor appear at a shallower place).
One way to solve this, apart from installing ruamel.yaml>=0.15.79
, is by
providing an alternative constructor, that forces depth first processing:
import sys
import ruamel.yaml
class MyConstructor(ruamel.yaml.constructor.RoundTripConstructor):
def construct_yaml_map(self, node):
data = ruamel.yaml.comments.CommentedMap()
data._yaml_set_line_col(node.start_mark.line, node.start_mark.column)
yield data
self.construct_mapping(node, data, deep=True)
self.set_collection_style(data, node)
MyConstructor.add_constructor(
u'tag:yaml.org,2002:map', MyConstructor.construct_yaml_map
)
yaml = ruamel.yaml.YAML()
yaml.Constructor = MyConstructor
yaml_str = """
Dict1:
- InnerDict: &inner
key: val
Dict2:
InnerDict:
<<: *inner
Dict3:
- InnerDict:
<<: *inner
"""
data = yaml.load(yaml_str)
for k in data:
print(k, data[k])
print('---------')
yaml.dump(data, sys.stdout)
Generating:
Dict1 [ordereddict([('InnerDict', ordereddict([('key', 'val')]))])]
Dict2 ordereddict([('InnerDict', ordereddict([('key', 'val')]))])
Dict3 [ordereddict([('InnerDict', ordereddict([('key', 'val')]))])]
---------
Dict1:
- InnerDict: &inner
key: val
Dict2:
InnerDict:
<<: *inner
Dict3:
- InnerDict:
<<: *inner
(As the dump of data
is correct even without the above "patch", this
was not previously detected when testing round-trips)
add a comment |
Your Answer
StackExchange.ifUsing("editor", function () {
StackExchange.using("externalEditor", function () {
StackExchange.using("snippets", function () {
StackExchange.snippets.init();
});
});
}, "code-snippets");
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "1"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53396845%2fwith-python-ruamel-yaml-lost-anchor-when-loading-in-round-trip-mode%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
That is definitely a bug, and it has to do with the breadth-first
construction of mappings in the data structure. By the time the
InnerDict
under Dict2
is constructed, the one nested under Dict1
is not completely available. The InnerDict
under Dict3
lies at the
same depth, and can therefore correctly be constructed (and so can the
first merge if you remove the dash in the Dict1
construction and so
making the anchor appear at a shallower place).
One way to solve this, apart from installing ruamel.yaml>=0.15.79
, is by
providing an alternative constructor, that forces depth first processing:
import sys
import ruamel.yaml
class MyConstructor(ruamel.yaml.constructor.RoundTripConstructor):
def construct_yaml_map(self, node):
data = ruamel.yaml.comments.CommentedMap()
data._yaml_set_line_col(node.start_mark.line, node.start_mark.column)
yield data
self.construct_mapping(node, data, deep=True)
self.set_collection_style(data, node)
MyConstructor.add_constructor(
u'tag:yaml.org,2002:map', MyConstructor.construct_yaml_map
)
yaml = ruamel.yaml.YAML()
yaml.Constructor = MyConstructor
yaml_str = """
Dict1:
- InnerDict: &inner
key: val
Dict2:
InnerDict:
<<: *inner
Dict3:
- InnerDict:
<<: *inner
"""
data = yaml.load(yaml_str)
for k in data:
print(k, data[k])
print('---------')
yaml.dump(data, sys.stdout)
Generating:
Dict1 [ordereddict([('InnerDict', ordereddict([('key', 'val')]))])]
Dict2 ordereddict([('InnerDict', ordereddict([('key', 'val')]))])
Dict3 [ordereddict([('InnerDict', ordereddict([('key', 'val')]))])]
---------
Dict1:
- InnerDict: &inner
key: val
Dict2:
InnerDict:
<<: *inner
Dict3:
- InnerDict:
<<: *inner
(As the dump of data
is correct even without the above "patch", this
was not previously detected when testing round-trips)
add a comment |
That is definitely a bug, and it has to do with the breadth-first
construction of mappings in the data structure. By the time the
InnerDict
under Dict2
is constructed, the one nested under Dict1
is not completely available. The InnerDict
under Dict3
lies at the
same depth, and can therefore correctly be constructed (and so can the
first merge if you remove the dash in the Dict1
construction and so
making the anchor appear at a shallower place).
One way to solve this, apart from installing ruamel.yaml>=0.15.79
, is by
providing an alternative constructor, that forces depth first processing:
import sys
import ruamel.yaml
class MyConstructor(ruamel.yaml.constructor.RoundTripConstructor):
def construct_yaml_map(self, node):
data = ruamel.yaml.comments.CommentedMap()
data._yaml_set_line_col(node.start_mark.line, node.start_mark.column)
yield data
self.construct_mapping(node, data, deep=True)
self.set_collection_style(data, node)
MyConstructor.add_constructor(
u'tag:yaml.org,2002:map', MyConstructor.construct_yaml_map
)
yaml = ruamel.yaml.YAML()
yaml.Constructor = MyConstructor
yaml_str = """
Dict1:
- InnerDict: &inner
key: val
Dict2:
InnerDict:
<<: *inner
Dict3:
- InnerDict:
<<: *inner
"""
data = yaml.load(yaml_str)
for k in data:
print(k, data[k])
print('---------')
yaml.dump(data, sys.stdout)
Generating:
Dict1 [ordereddict([('InnerDict', ordereddict([('key', 'val')]))])]
Dict2 ordereddict([('InnerDict', ordereddict([('key', 'val')]))])
Dict3 [ordereddict([('InnerDict', ordereddict([('key', 'val')]))])]
---------
Dict1:
- InnerDict: &inner
key: val
Dict2:
InnerDict:
<<: *inner
Dict3:
- InnerDict:
<<: *inner
(As the dump of data
is correct even without the above "patch", this
was not previously detected when testing round-trips)
add a comment |
That is definitely a bug, and it has to do with the breadth-first
construction of mappings in the data structure. By the time the
InnerDict
under Dict2
is constructed, the one nested under Dict1
is not completely available. The InnerDict
under Dict3
lies at the
same depth, and can therefore correctly be constructed (and so can the
first merge if you remove the dash in the Dict1
construction and so
making the anchor appear at a shallower place).
One way to solve this, apart from installing ruamel.yaml>=0.15.79
, is by
providing an alternative constructor, that forces depth first processing:
import sys
import ruamel.yaml
class MyConstructor(ruamel.yaml.constructor.RoundTripConstructor):
def construct_yaml_map(self, node):
data = ruamel.yaml.comments.CommentedMap()
data._yaml_set_line_col(node.start_mark.line, node.start_mark.column)
yield data
self.construct_mapping(node, data, deep=True)
self.set_collection_style(data, node)
MyConstructor.add_constructor(
u'tag:yaml.org,2002:map', MyConstructor.construct_yaml_map
)
yaml = ruamel.yaml.YAML()
yaml.Constructor = MyConstructor
yaml_str = """
Dict1:
- InnerDict: &inner
key: val
Dict2:
InnerDict:
<<: *inner
Dict3:
- InnerDict:
<<: *inner
"""
data = yaml.load(yaml_str)
for k in data:
print(k, data[k])
print('---------')
yaml.dump(data, sys.stdout)
Generating:
Dict1 [ordereddict([('InnerDict', ordereddict([('key', 'val')]))])]
Dict2 ordereddict([('InnerDict', ordereddict([('key', 'val')]))])
Dict3 [ordereddict([('InnerDict', ordereddict([('key', 'val')]))])]
---------
Dict1:
- InnerDict: &inner
key: val
Dict2:
InnerDict:
<<: *inner
Dict3:
- InnerDict:
<<: *inner
(As the dump of data
is correct even without the above "patch", this
was not previously detected when testing round-trips)
That is definitely a bug, and it has to do with the breadth-first
construction of mappings in the data structure. By the time the
InnerDict
under Dict2
is constructed, the one nested under Dict1
is not completely available. The InnerDict
under Dict3
lies at the
same depth, and can therefore correctly be constructed (and so can the
first merge if you remove the dash in the Dict1
construction and so
making the anchor appear at a shallower place).
One way to solve this, apart from installing ruamel.yaml>=0.15.79
, is by
providing an alternative constructor, that forces depth first processing:
import sys
import ruamel.yaml
class MyConstructor(ruamel.yaml.constructor.RoundTripConstructor):
def construct_yaml_map(self, node):
data = ruamel.yaml.comments.CommentedMap()
data._yaml_set_line_col(node.start_mark.line, node.start_mark.column)
yield data
self.construct_mapping(node, data, deep=True)
self.set_collection_style(data, node)
MyConstructor.add_constructor(
u'tag:yaml.org,2002:map', MyConstructor.construct_yaml_map
)
yaml = ruamel.yaml.YAML()
yaml.Constructor = MyConstructor
yaml_str = """
Dict1:
- InnerDict: &inner
key: val
Dict2:
InnerDict:
<<: *inner
Dict3:
- InnerDict:
<<: *inner
"""
data = yaml.load(yaml_str)
for k in data:
print(k, data[k])
print('---------')
yaml.dump(data, sys.stdout)
Generating:
Dict1 [ordereddict([('InnerDict', ordereddict([('key', 'val')]))])]
Dict2 ordereddict([('InnerDict', ordereddict([('key', 'val')]))])
Dict3 [ordereddict([('InnerDict', ordereddict([('key', 'val')]))])]
---------
Dict1:
- InnerDict: &inner
key: val
Dict2:
InnerDict:
<<: *inner
Dict3:
- InnerDict:
<<: *inner
(As the dump of data
is correct even without the above "patch", this
was not previously detected when testing round-trips)
edited Nov 21 '18 at 7:45
answered Nov 20 '18 at 16:50
AnthonAnthon
30.7k1795148
30.7k1795148
add a comment |
add a comment |
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53396845%2fwith-python-ruamel-yaml-lost-anchor-when-loading-in-round-trip-mode%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown