Processing a large dataset of nested data
.everyoneloves__top-leaderboard:empty,.everyoneloves__mid-leaderboard:empty,.everyoneloves__bot-mid-leaderboard:empty{ height:90px;width:728px;box-sizing:border-box;
}
I have a rather large set of data which is structured in a somewhat unique fashion. It looks something like this:
foo:
- name: "some name"
location: "some location"
type: "someType"
bar:
- name: "A bar element"
location: "location here"
type: "someOtherType"
attachments:
- type: "attachmentTypeA"
name: "Attachment name"
- type: "attachmentTypeB"
name: "Attachment name"
baz:
- name: "another name"
location: "another location"
type: "anotherType"
qux:
- name: "My name here"
location: "My location here"
type: "SomeOtherTypeHere"
xyzzy:
- name: "Another name here"
location: "Another location here"
type: "anotherTypeHere"
bar:
- name: "Some name here"
location: "Some location here"
type: "typeHere"
attachments:
- type: "attachmentTypeA"
name: "attachment name here"
- type: "attachmentTypeA"
name: "attachment name here"
- type: "attachmentTypeB"
name: "attachment name here"
- name: "Another name here"
location: "Another location here"
type: "anotherTypeHere"
attachments:
- type: "attachmentTypeA"
name: "attachment name here"
- type: "attachmentTypeC"
name: "attachment name here"
- type: "attachmentTypeD"
name: "attachment name here"
- name: "Another baz listing"
location: "Baz location"
type: "bazTypeHere"
So basically, you have "foo" at the top level (and there can be more than one foo, but always at the top level). In general, the structure is:
foo > baz > qux > xyzzy > bar
However, any of the sub elements can be at the root, or under foo, provided they are in order. So these are valid:
foo
qux
xyzzy
bar
attachments
bar
attachments
As is this:
foo
baz
qux
xyzzy
bar
attachments
bar
attachments
qux
xyzzy
bar
attachments
bar
attachments
xyzzy
bar
attachments
bar
attachments
And so on. It's whacky, I know. But that's the dataset I inherited. I looked at the examples, in particular the DeserializeObjectGraph and LoadingAYamlStream examples. The DeserializeObjectGraph approach gets kind of crazy when the data is laid out like this. I finally gave up on it as it just got too hairy. The stream approach seems like a better fit, I think, but I'm running into troubles.
I am loading up the YAML as follows:
string contents = System.IO.File.ReadAllText ( fileName );
var input = new StringReader (contents);
var yaml = new YamlStream ();
yaml.Load (input);
As you can see, nothing fancy there. I'm just trying to get a "tree" of objects that I can then iterate through. I tried using the AllNodes property from the root node, but I can't for the life of me figure out how to iterate through them recursively in some manner than makes sense. I will also confess that I am a C# n00btard that is still learning (old C guy here), so bear with me!
Can anyone suggest an approach, or possibly some code or even pseudocode that might be able to help me out?
yamldotnet
asked Nov 22 '18 at 4:24
incursioincursio
11
|
show 2 more comments
I have a rather large set of data which is structured in a somewhat unique fashion. It looks something like this:
foo:
- name: "some name"
location: "some location"
type: "someType"
bar:
- name: "A bar element"
location: "location here"
type: "someOtherType"
attachments:
- type: "attachmentTypeA"
name: "Attachment name"
- type: "attachmentTypeB"
name: "Attachment name"
baz:
- name: "another name"
location: "another location"
type: "anotherType"
qux:
- name: "My name here"
location: "My location here"
type: "SomeOtherTypeHere"
xyzzy:
- name: "Another name here"
location: "Another location here"
type: "anotherTypeHere"
bar:
- name: "Some name here"
location: "Some location here"
type: "typeHere"
attachments:
- type: "attachmentTypeA"
name: "attachment name here"
- type: "attachmentTypeA"
name: "attachment name here"
- type: "attachmentTypeB"
name: "attachment name here"
- name: "Another name here"
location: "Another location here"
type: "anotherTypeHere"
attachments:
- type: "attachmentTypeA"
name: "attachment name here"
- type: "attachmentTypeC"
name: "attachment name here"
- type: "attachmentTypeD"
name: "attachment name here"
- name: "Another baz listing"
location: "Baz location"
type: "bazTypeHere"
So basically, you have "foo" at the top level (and there can be more than one foo, but always at the top level). In general, the structure is:
foo > baz > qux > xyzzy > bar
However, any of the sub elements can be at the root, or under foo, provided they are in order. So these are valid:
foo
qux
xyzzy
bar
attachments
bar
attachments
As is this:
foo
baz
qux
xyzzy
bar
attachments
bar
attachments
qux
xyzzy
bar
attachments
bar
attachments
xyzzy
bar
attachments
bar
attachments
And so on. It's whacky, I know. But that's the dataset I inherited. I looked at the examples, in particular the DeserializeObjectGraph and LoadingAYamlStream examples. The DeserializeObjectGraph approach gets kind of crazy when the data is laid out like this. I finally gave up on it as it just got too hairy. The stream approach seems like a better fit, I think, but I'm running into troubles.
I am loading up the YAML as follows:
string contents = System.IO.File.ReadAllText ( fileName );
var input = new StringReader (contents);
var yaml = new YamlStream ();
yaml.Load (input);
As you can see, nothing fancy there. I'm just trying to get a "tree" of objects that I can then iterate through. I tried using the AllNodes property from the root node, but I can't for the life of me figure out how to iterate through them recursively in some manner than makes sense. I will also confess that I am a C# n00btard that is still learning (old C guy here), so bear with me!
Can anyone suggest an approach, or possibly some code or even pseudocode that might be able to help me out?
yamldotnet
asked Nov 22 '18 at 4:24
incursioincursio
11
What is your goal, what kind of output do you expect from any code? E.G. I imagine you could use javascript in the browser to iterate through objects - it wouldn't take long to write that.
– zyrup
Nov 22 '18 at 4:36
This is using YamlDotNet under Unity, so no browser involved. As I said, ideally, I would process it with YamlDotNet and end up with a tree that I could then sift through and work with. If that makes any sense.
– incursio
Nov 22 '18 at 5:23
Can you add the code that you used to deserialize the document ? This would help providing suggestions.
– Antoine Aubry
Nov 22 '18 at 14:54
Antoine, thanks for the quick response! I added a code snippet and a bit more flavor to my original post. As you can see from it, there is nothing fancy there really. I just load the markup from a file. Now I just need to figure out how to iterate through them. That's the part I'm struggling with.
– incursio
Nov 22 '18 at 16:01
I should add, it isn't so much about the data structure I'm dealing with as it is I guess just how to iterate through them at a high level. I can recursively do it, if need be, just need to figure out the right syntax/constructs involved.
– incursio
Nov 22 '18 at 17:14
|
show 2 more comments
I have a rather large set of data which is structured in a somewhat unique fashion. It looks something like this:
foo:
- name: "some name"
location: "some location"
type: "someType"
bar:
- name: "A bar element"
location: "location here"
type: "someOtherType"
attachments:
- type: "attachmentTypeA"
name: "Attachment name"
- type: "attachmentTypeB"
name: "Attachment name"
baz:
- name: "another name"
location: "another location"
type: "anotherType"
qux:
- name: "My name here"
location: "My location here"
type: "SomeOtherTypeHere"
xyzzy:
- name: "Another name here"
location: "Another location here"
type: "anotherTypeHere"
bar:
- name: "Some name here"
location: "Some location here"
type: "typeHere"
attachments:
- type: "attachmentTypeA"
name: "attachment name here"
- type: "attachmentTypeA"
name: "attachment name here"
- type: "attachmentTypeB"
name: "attachment name here"
- name: "Another name here"
location: "Another location here"
type: "anotherTypeHere"
attachments:
- type: "attachmentTypeA"
name: "attachment name here"
- type: "attachmentTypeC"
name: "attachment name here"
- type: "attachmentTypeD"
name: "attachment name here"
- name: "Another baz listing"
location: "Baz location"
type: "bazTypeHere"
So basically, you have "foo" at the top level (and there can be more than one foo, but always at the top level). In general, the structure is:
foo > baz > qux > xyzzy > bar
However, any of the sub elements can be at the root, or under foo, provided they are in order. So these are valid:
foo
qux
xyzzy
bar
attachments
bar
attachments
As is this:
foo
baz
qux
xyzzy
bar
attachments
bar
attachments
qux
xyzzy
bar
attachments
bar
attachments
xyzzy
bar
attachments
bar
attachments
And so on. It's whacky, I know. But that's the dataset I inherited. I looked at the examples, in particular the DeserializeObjectGraph and LoadingAYamlStream examples. The DeserializeObjectGraph approach gets kind of crazy when the data is laid out like this. I finally gave up on it as it just got too hairy. The stream approach seems like a better fit, I think, but I'm running into troubles.
I am loading up the YAML as follows:
string contents = System.IO.File.ReadAllText ( fileName );
var input = new StringReader (contents);
var yaml = new YamlStream ();
yaml.Load (input);
As you can see, nothing fancy there. I'm just trying to get a "tree" of objects that I can then iterate through. I tried using the AllNodes property from the root node, but I can't for the life of me figure out how to iterate through them recursively in some manner than makes sense. I will also confess that I am a C# n00btard that is still learning (old C guy here), so bear with me!
Can anyone suggest an approach, or possibly some code or even pseudocode that might be able to help me out?
yamldotnet
asked Nov 22 '18 at 4:24
incursioincursio
11
I have a rather large set of data which is structured in a somewhat unique fashion. It looks something like this:
foo:
- name: "some name"
location: "some location"
type: "someType"
bar:
- name: "A bar element"
location: "location here"
type: "someOtherType"
attachments:
- type: "attachmentTypeA"
name: "Attachment name"
- type: "attachmentTypeB"
name: "Attachment name"
baz:
- name: "another name"
location: "another location"
type: "anotherType"
qux:
- name: "My name here"
location: "My location here"
type: "SomeOtherTypeHere"
xyzzy:
- name: "Another name here"
location: "Another location here"
type: "anotherTypeHere"
bar:
- name: "Some name here"
location: "Some location here"
type: "typeHere"
attachments:
- type: "attachmentTypeA"
name: "attachment name here"
- type: "attachmentTypeA"
name: "attachment name here"
- type: "attachmentTypeB"
name: "attachment name here"
- name: "Another name here"
location: "Another location here"
type: "anotherTypeHere"
attachments:
- type: "attachmentTypeA"
name: "attachment name here"
- type: "attachmentTypeC"
name: "attachment name here"
- type: "attachmentTypeD"
name: "attachment name here"
- name: "Another baz listing"
location: "Baz location"
type: "bazTypeHere"
So basically, you have "foo" at the top level (and there can be more than one foo, but always at the top level). In general, the structure is:
foo > baz > qux > xyzzy > bar
However, any of the sub elements can be at the root, or under foo, provided they are in order. So these are valid:
foo
qux
xyzzy
bar
attachments
bar
attachments
As is this:
foo
baz
qux
xyzzy
bar
attachments
bar
attachments
qux
xyzzy
bar
attachments
bar
attachments
xyzzy
bar
attachments
bar
attachments
And so on. It's whacky, I know. But that's the dataset I inherited. I looked at the examples, in particular the DeserializeObjectGraph and LoadingAYamlStream examples. The DeserializeObjectGraph approach gets kind of crazy when the data is laid out like this. I finally gave up on it as it just got too hairy. The stream approach seems like a better fit, I think, but I'm running into troubles.
I am loading up the YAML as follows:
string contents = System.IO.File.ReadAllText ( fileName );
var input = new StringReader (contents);
var yaml = new YamlStream ();
yaml.Load (input);
As you can see, nothing fancy there. I'm just trying to get a "tree" of objects that I can then iterate through. I tried using the AllNodes property from the root node, but I can't for the life of me figure out how to iterate through them recursively in some manner than makes sense. I will also confess that I am a C# n00btard that is still learning (old C guy here), so bear with me!
Can anyone suggest an approach, or possibly some code or even pseudocode that might be able to help me out?
yamldotnet
yamldotnet
asked Nov 22 '18 at 4:24
incursioincursio
11
asked Nov 22 '18 at 4:24
incursioincursio
11
edited Nov 22 '18 at 16:00
incursio
asked Nov 22 '18 at 4:24
incursioincursio
11
asked Nov 22 '18 at 4:24
incursioincursio
11
asked Nov 22 '18 at 4:24
incursioincursio
11
11
What is your goal, what kind of output do you expect from any code? E.G. I imagine you could use javascript in the browser to iterate through objects - it wouldn't take long to write that.
– zyrup
Nov 22 '18 at 4:36
This is using YamlDotNet under Unity, so no browser involved. As I said, ideally, I would process it with YamlDotNet and end up with a tree that I could then sift through and work with. If that makes any sense.
– incursio
Nov 22 '18 at 5:23
Can you add the code that you used to deserialize the document ? This would help providing suggestions.
– Antoine Aubry
Nov 22 '18 at 14:54
Antoine, thanks for the quick response! I added a code snippet and a bit more flavor to my original post. As you can see from it, there is nothing fancy there really. I just load the markup from a file. Now I just need to figure out how to iterate through them. That's the part I'm struggling with.
– incursio
Nov 22 '18 at 16:01
I should add, it isn't so much about the data structure I'm dealing with as it is I guess just how to iterate through them at a high level. I can recursively do it, if need be, just need to figure out the right syntax/constructs involved.
– incursio
Nov 22 '18 at 17:14
|
show 2 more comments
What is your goal, what kind of output do you expect from any code? E.G. I imagine you could use javascript in the browser to iterate through objects - it wouldn't take long to write that.
– zyrup
Nov 22 '18 at 4:36
This is using YamlDotNet under Unity, so no browser involved. As I said, ideally, I would process it with YamlDotNet and end up with a tree that I could then sift through and work with. If that makes any sense.
– incursio
Nov 22 '18 at 5:23
Can you add the code that you used to deserialize the document ? This would help providing suggestions.
– Antoine Aubry
Nov 22 '18 at 14:54
Antoine, thanks for the quick response! I added a code snippet and a bit more flavor to my original post. As you can see from it, there is nothing fancy there really. I just load the markup from a file. Now I just need to figure out how to iterate through them. That's the part I'm struggling with.
– incursio
Nov 22 '18 at 16:01
I should add, it isn't so much about the data structure I'm dealing with as it is I guess just how to iterate through them at a high level. I can recursively do it, if need be, just need to figure out the right syntax/constructs involved.
– incursio
Nov 22 '18 at 17:14
What is your goal, what kind of output do you expect from any code? E.G. I imagine you could use javascript in the browser to iterate through objects - it wouldn't take long to write that.
– zyrup
Nov 22 '18 at 4:36
What is your goal, what kind of output do you expect from any code? E.G. I imagine you could use javascript in the browser to iterate through objects - it wouldn't take long to write that.
– zyrup
Nov 22 '18 at 4:36
This is using YamlDotNet under Unity, so no browser involved. As I said, ideally, I would process it with YamlDotNet and end up with a tree that I could then sift through and work with. If that makes any sense.
– incursio
Nov 22 '18 at 5:23
This is using YamlDotNet under Unity, so no browser involved. As I said, ideally, I would process it with YamlDotNet and end up with a tree that I could then sift through and work with. If that makes any sense.
– incursio
Nov 22 '18 at 5:23
Can you add the code that you used to deserialize the document ? This would help providing suggestions.
– Antoine Aubry
Nov 22 '18 at 14:54
Can you add the code that you used to deserialize the document ? This would help providing suggestions.
– Antoine Aubry
Nov 22 '18 at 14:54
Antoine, thanks for the quick response! I added a code snippet and a bit more flavor to my original post. As you can see from it, there is nothing fancy there really. I just load the markup from a file. Now I just need to figure out how to iterate through them. That's the part I'm struggling with.
– incursio
Nov 22 '18 at 16:01
Antoine, thanks for the quick response! I added a code snippet and a bit more flavor to my original post. As you can see from it, there is nothing fancy there really. I just load the markup from a file. Now I just need to figure out how to iterate through them. That's the part I'm struggling with.
– incursio
Nov 22 '18 at 16:01
I should add, it isn't so much about the data structure I'm dealing with as it is I guess just how to iterate through them at a high level. I can recursively do it, if need be, just need to figure out the right syntax/constructs involved.
– incursio
Nov 22 '18 at 17:14
I should add, it isn't so much about the data structure I'm dealing with as it is I guess just how to iterate through them at a high level. I can recursively do it, if need be, just need to figure out the right syntax/constructs involved.
– incursio
Nov 22 '18 at 17:14
|
show 2 more comments
0
active
oldest
votes
Your Answer
StackExchange.ifUsing("editor", function () {
StackExchange.using("externalEditor", function () {
StackExchange.using("snippets", function () {
StackExchange.snippets.init();
});
});
}, "code-snippets");
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "1"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53423874%2fprocessing-a-large-dataset-of-nested-data%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
0
active
oldest
votes
0
active
oldest
votes
active
oldest
votes
active
oldest
votes
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53423874%2fprocessing-a-large-dataset-of-nested-data%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
What is your goal, what kind of output do you expect from any code? E.G. I imagine you could use javascript in the browser to iterate through objects - it wouldn't take long to write that.
– zyrup
Nov 22 '18 at 4:36
This is using YamlDotNet under Unity, so no browser involved. As I said, ideally, I would process it with YamlDotNet and end up with a tree that I could then sift through and work with. If that makes any sense.
– incursio
Nov 22 '18 at 5:23
Can you add the code that you used to deserialize the document ? This would help providing suggestions.
– Antoine Aubry
Nov 22 '18 at 14:54
Antoine, thanks for the quick response! I added a code snippet and a bit more flavor to my original post. As you can see from it, there is nothing fancy there really. I just load the markup from a file. Now I just need to figure out how to iterate through them. That's the part I'm struggling with.
– incursio
Nov 22 '18 at 16:01
I should add, it isn't so much about the data structure I'm dealing with as it is I guess just how to iterate through them at a high level. I can recursively do it, if need be, just need to figure out the right syntax/constructs involved.
– incursio
Nov 22 '18 at 17:14