How do I sum the values in an array of maps in jq?
.everyoneloves__top-leaderboard:empty,.everyoneloves__mid-leaderboard:empty,.everyoneloves__bot-mid-leaderboard:empty{ height:90px;width:728px;box-sizing:border-box;
}
Given a JSON stream of the following form:
{ "a": 10, "b": 11 } { "a": 20, "b": 21 } { "a": 30, "b": 31 }
I would like to sum the values in each of the objects and output a single object, namely:
{ "a": 60, "b": 63 }
I'm guessing this will probably require flattening the above list of objects into a an array of [name, value] pairs and then summing the values using reduce but the documentation of the syntax for using reduce is woeful.
json sum key jq
edited Nov 24 '17 at 18:13
peak
34.7k94461
asked Feb 12 '15 at 17:59
Alan BurlisonAlan Burlison
6471616
add a comment |
Given a JSON stream of the following form:
{ "a": 10, "b": 11 } { "a": 20, "b": 21 } { "a": 30, "b": 31 }
I would like to sum the values in each of the objects and output a single object, namely:
{ "a": 60, "b": 63 }
I'm guessing this will probably require flattening the above list of objects into a an array of [name, value] pairs and then summing the values using reduce but the documentation of the syntax for using reduce is woeful.
json sum key jq
edited Nov 24 '17 at 18:13
peak
34.7k94461
asked Feb 12 '15 at 17:59
Alan BurlisonAlan Burlison
6471616
add a comment |
Given a JSON stream of the following form:
{ "a": 10, "b": 11 } { "a": 20, "b": 21 } { "a": 30, "b": 31 }
I would like to sum the values in each of the objects and output a single object, namely:
{ "a": 60, "b": 63 }
I'm guessing this will probably require flattening the above list of objects into a an array of [name, value] pairs and then summing the values using reduce but the documentation of the syntax for using reduce is woeful.
json sum key jq
edited Nov 24 '17 at 18:13
peak
34.7k94461
asked Feb 12 '15 at 17:59
Alan BurlisonAlan Burlison
6471616
Given a JSON stream of the following form:
{ "a": 10, "b": 11 } { "a": 20, "b": 21 } { "a": 30, "b": 31 }
I would like to sum the values in each of the objects and output a single object, namely:
{ "a": 60, "b": 63 }
I'm guessing this will probably require flattening the above list of objects into a an array of [name, value] pairs and then summing the values using reduce but the documentation of the syntax for using reduce is woeful.
json sum key jq
json sum key jq
edited Nov 24 '17 at 18:13
peak
34.7k94461
asked Feb 12 '15 at 17:59
Alan BurlisonAlan Burlison
6471616
edited Nov 24 '17 at 18:13
peak
34.7k94461
asked Feb 12 '15 at 17:59
Alan BurlisonAlan Burlison
6471616
edited Nov 24 '17 at 18:13
peak
34.7k94461
edited Nov 24 '17 at 18:13
peak
34.7k94461
edited Nov 24 '17 at 18:13
peak
34.7k94461
34.7k94461
asked Feb 12 '15 at 17:59
Alan BurlisonAlan Burlison
6471616
asked Feb 12 '15 at 17:59
Alan BurlisonAlan Burlison
6471616
asked Feb 12 '15 at 17:59
Alan BurlisonAlan Burlison
6471616
6471616
add a comment |
add a comment |
2 Answers
2
active
oldest
votes
Unless your jq has inputs, you will have to slurp the objects up using the -s flag. Then you'll have to do a fair amount of manipulation:
- Each of the objects needs to be mapped out to key/value pairs
- Flatten the pairs to a single array
- Group up the pairs by key
- Map out each group accumulating the values to a single key/value pair
- Map the pairs back to an object
map(to_entries)
| add
| group_by(.key)
| map({
key: .[0].key,
value: map(.value) | add
})
| from_entries
With jq 1.5, this could be greatly improved: You can do away with slurping and just read the inputs directly.
$ jq -n '
reduce (inputs | to_entries) as {$key,$value} ({}; .[$key] += $value)
' input.json
Since we're simply accumulating all the values in each of the objects, it'll be easier to just run through the key/value pairs of all the inputs, and add them all up.
edited Dec 27 '18 at 10:02
peak
34.7k94461
answered Feb 12 '15 at 18:52
Jeff MercadoJeff Mercado
93.6k20188216
Ah, the group_by is what I was missing, I'd overlooked it in the documentation. Also, I got the tool producing the source data to output as a single JSON stream so '-s' wasn't needed. The only other tweak I made was to use 'flatten' instead of 'add'. Thanks for the answer, it was spot on :-)
– Alan Burlison
Feb 12 '15 at 22:00
2
I stand in awe.
– ssc
Apr 10 '16 at 11:46
Reduce is awesome! I'm struggling with how it might support 2 methods (sum and avg for isntance) It doesnt like sem-colons, commas seem to run the last manipulation only reduce .usage."os:linux" as $item ( {"credits":0,"minutes":0}; ."credits" += $item.credits; ."minutes" += $item.amount /1000/60 )
– Eddie
Nov 21 '18 at 19:55
@Eddie: semicolons are used for separating parameters to function calls. The way you're using it there, it should be pipes and it would work like you would expect.
– Jeff Mercado
Nov 22 '18 at 7:48
add a comment |
Another approach, which illustrates the power of jq quite nicely, is to use a filter named "sum" defined as follows:
def sum(f): reduce . as $row (0; . + ($row|f) );
To solve the particular problem at hand, one could then use the -s (--slurp) option as mentioned above, together with the expression:
{"a": sum(.a), "b": sum(.b) } # (2)
The expression labeled (2) only computes the two specified sums, but it is easy to generalize, e.g. as follows:
# Produce an object with the same keys as the first object in the
# input array, but with values equal to the sum of the corresponding
# values in all the objects.
def sumByKey:
. as $in
| reduce (.[0] | keys) as $key
( {}; . + {($key): ($in | sum(.[$key]))})
;
answered Feb 16 '15 at 6:08
peakpeak
34.7k94461
add a comment |
Your Answer
StackExchange.ifUsing("editor", function () {
StackExchange.using("externalEditor", function () {
StackExchange.using("snippets", function () {
StackExchange.snippets.init();
});
});
}, "code-snippets");
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "1"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f28484534%2fhow-do-i-sum-the-values-in-an-array-of-maps-in-jq%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
2 Answers
2
active
oldest
votes
2 Answers
2
active
oldest
votes
active
oldest
votes
active
oldest
votes
Unless your jq has inputs, you will have to slurp the objects up using the -s flag. Then you'll have to do a fair amount of manipulation:
- Each of the objects needs to be mapped out to key/value pairs
- Flatten the pairs to a single array
- Group up the pairs by key
- Map out each group accumulating the values to a single key/value pair
- Map the pairs back to an object
map(to_entries)
| add
| group_by(.key)
| map({
key: .[0].key,
value: map(.value) | add
})
| from_entries
With jq 1.5, this could be greatly improved: You can do away with slurping and just read the inputs directly.
$ jq -n '
reduce (inputs | to_entries) as {$key,$value} ({}; .[$key] += $value)
' input.json
Since we're simply accumulating all the values in each of the objects, it'll be easier to just run through the key/value pairs of all the inputs, and add them all up.
edited Dec 27 '18 at 10:02
peak
34.7k94461
answered Feb 12 '15 at 18:52
Jeff MercadoJeff Mercado
93.6k20188216
Ah, the group_by is what I was missing, I'd overlooked it in the documentation. Also, I got the tool producing the source data to output as a single JSON stream so '-s' wasn't needed. The only other tweak I made was to use 'flatten' instead of 'add'. Thanks for the answer, it was spot on :-)
– Alan Burlison
Feb 12 '15 at 22:00
2
I stand in awe.
– ssc
Apr 10 '16 at 11:46
Reduce is awesome! I'm struggling with how it might support 2 methods (sum and avg for isntance) It doesnt like sem-colons, commas seem to run the last manipulation only reduce .usage."os:linux" as $item ( {"credits":0,"minutes":0}; ."credits" += $item.credits; ."minutes" += $item.amount /1000/60 )
– Eddie
Nov 21 '18 at 19:55
@Eddie: semicolons are used for separating parameters to function calls. The way you're using it there, it should be pipes and it would work like you would expect.
– Jeff Mercado
Nov 22 '18 at 7:48
add a comment |
Unless your jq has inputs, you will have to slurp the objects up using the -s flag. Then you'll have to do a fair amount of manipulation:
- Each of the objects needs to be mapped out to key/value pairs
- Flatten the pairs to a single array
- Group up the pairs by key
- Map out each group accumulating the values to a single key/value pair
- Map the pairs back to an object
map(to_entries)
| add
| group_by(.key)
| map({
key: .[0].key,
value: map(.value) | add
})
| from_entries
With jq 1.5, this could be greatly improved: You can do away with slurping and just read the inputs directly.
$ jq -n '
reduce (inputs | to_entries) as {$key,$value} ({}; .[$key] += $value)
' input.json
Since we're simply accumulating all the values in each of the objects, it'll be easier to just run through the key/value pairs of all the inputs, and add them all up.
edited Dec 27 '18 at 10:02
peak
34.7k94461
answered Feb 12 '15 at 18:52
Jeff MercadoJeff Mercado
93.6k20188216
Ah, the group_by is what I was missing, I'd overlooked it in the documentation. Also, I got the tool producing the source data to output as a single JSON stream so '-s' wasn't needed. The only other tweak I made was to use 'flatten' instead of 'add'. Thanks for the answer, it was spot on :-)
– Alan Burlison
Feb 12 '15 at 22:00
2
I stand in awe.
– ssc
Apr 10 '16 at 11:46
Reduce is awesome! I'm struggling with how it might support 2 methods (sum and avg for isntance) It doesnt like sem-colons, commas seem to run the last manipulation only reduce .usage."os:linux" as $item ( {"credits":0,"minutes":0}; ."credits" += $item.credits; ."minutes" += $item.amount /1000/60 )
– Eddie
Nov 21 '18 at 19:55
@Eddie: semicolons are used for separating parameters to function calls. The way you're using it there, it should be pipes and it would work like you would expect.
– Jeff Mercado
Nov 22 '18 at 7:48
add a comment |
Unless your jq has inputs, you will have to slurp the objects up using the -s flag. Then you'll have to do a fair amount of manipulation:
- Each of the objects needs to be mapped out to key/value pairs
- Flatten the pairs to a single array
- Group up the pairs by key
- Map out each group accumulating the values to a single key/value pair
- Map the pairs back to an object
map(to_entries)
| add
| group_by(.key)
| map({
key: .[0].key,
value: map(.value) | add
})
| from_entries
With jq 1.5, this could be greatly improved: You can do away with slurping and just read the inputs directly.
$ jq -n '
reduce (inputs | to_entries) as {$key,$value} ({}; .[$key] += $value)
' input.json
Since we're simply accumulating all the values in each of the objects, it'll be easier to just run through the key/value pairs of all the inputs, and add them all up.
edited Dec 27 '18 at 10:02
peak
34.7k94461
answered Feb 12 '15 at 18:52
Jeff MercadoJeff Mercado
93.6k20188216
Unless your jq has inputs, you will have to slurp the objects up using the -s flag. Then you'll have to do a fair amount of manipulation:
- Each of the objects needs to be mapped out to key/value pairs
- Flatten the pairs to a single array
- Group up the pairs by key
- Map out each group accumulating the values to a single key/value pair
- Map the pairs back to an object
map(to_entries)
| add
| group_by(.key)
| map({
key: .[0].key,
value: map(.value) | add
})
| from_entries
With jq 1.5, this could be greatly improved: You can do away with slurping and just read the inputs directly.
$ jq -n '
reduce (inputs | to_entries) as {$key,$value} ({}; .[$key] += $value)
' input.json
Since we're simply accumulating all the values in each of the objects, it'll be easier to just run through the key/value pairs of all the inputs, and add them all up.
edited Dec 27 '18 at 10:02
peak
34.7k94461
answered Feb 12 '15 at 18:52
Jeff MercadoJeff Mercado
93.6k20188216
edited Dec 27 '18 at 10:02
peak
34.7k94461
edited Dec 27 '18 at 10:02
peak
34.7k94461
edited Dec 27 '18 at 10:02
peak
34.7k94461
34.7k94461
answered Feb 12 '15 at 18:52
Jeff MercadoJeff Mercado
93.6k20188216
answered Feb 12 '15 at 18:52
Jeff MercadoJeff Mercado
93.6k20188216
answered Feb 12 '15 at 18:52
Jeff MercadoJeff Mercado
93.6k20188216
93.6k20188216
Ah, the group_by is what I was missing, I'd overlooked it in the documentation. Also, I got the tool producing the source data to output as a single JSON stream so '-s' wasn't needed. The only other tweak I made was to use 'flatten' instead of 'add'. Thanks for the answer, it was spot on :-)
– Alan Burlison
Feb 12 '15 at 22:00
2
I stand in awe.
– ssc
Apr 10 '16 at 11:46
Reduce is awesome! I'm struggling with how it might support 2 methods (sum and avg for isntance) It doesnt like sem-colons, commas seem to run the last manipulation only reduce .usage."os:linux" as $item ( {"credits":0,"minutes":0}; ."credits" += $item.credits; ."minutes" += $item.amount /1000/60 )
– Eddie
Nov 21 '18 at 19:55
@Eddie: semicolons are used for separating parameters to function calls. The way you're using it there, it should be pipes and it would work like you would expect.
– Jeff Mercado
Nov 22 '18 at 7:48
add a comment |
Ah, the group_by is what I was missing, I'd overlooked it in the documentation. Also, I got the tool producing the source data to output as a single JSON stream so '-s' wasn't needed. The only other tweak I made was to use 'flatten' instead of 'add'. Thanks for the answer, it was spot on :-)
– Alan Burlison
Feb 12 '15 at 22:00
2
I stand in awe.
– ssc
Apr 10 '16 at 11:46
Reduce is awesome! I'm struggling with how it might support 2 methods (sum and avg for isntance) It doesnt like sem-colons, commas seem to run the last manipulation only reduce .usage."os:linux" as $item ( {"credits":0,"minutes":0}; ."credits" += $item.credits; ."minutes" += $item.amount /1000/60 )
– Eddie
Nov 21 '18 at 19:55
@Eddie: semicolons are used for separating parameters to function calls. The way you're using it there, it should be pipes and it would work like you would expect.
– Jeff Mercado
Nov 22 '18 at 7:48
Ah, the group_by is what I was missing, I'd overlooked it in the documentation. Also, I got the tool producing the source data to output as a single JSON stream so '-s' wasn't needed. The only other tweak I made was to use 'flatten' instead of 'add'. Thanks for the answer, it was spot on :-)
– Alan Burlison
Feb 12 '15 at 22:00
Ah, the group_by is what I was missing, I'd overlooked it in the documentation. Also, I got the tool producing the source data to output as a single JSON stream so '-s' wasn't needed. The only other tweak I made was to use 'flatten' instead of 'add'. Thanks for the answer, it was spot on :-)
– Alan Burlison
Feb 12 '15 at 22:00
2
2
I stand in awe.
– ssc
Apr 10 '16 at 11:46
I stand in awe.
– ssc
Apr 10 '16 at 11:46
Reduce is awesome! I'm struggling with how it might support 2 methods (sum and avg for isntance) It doesnt like sem-colons, commas seem to run the last manipulation only reduce .usage."os:linux" as $item ( {"credits":0,"minutes":0}; ."credits" += $item.credits; ."minutes" += $item.amount /1000/60 )
– Eddie
Nov 21 '18 at 19:55
Reduce is awesome! I'm struggling with how it might support 2 methods (sum and avg for isntance) It doesnt like sem-colons, commas seem to run the last manipulation only reduce .usage."os:linux" as $item ( {"credits":0,"minutes":0}; ."credits" += $item.credits; ."minutes" += $item.amount /1000/60 )
– Eddie
Nov 21 '18 at 19:55
@Eddie: semicolons are used for separating parameters to function calls. The way you're using it there, it should be pipes and it would work like you would expect.
– Jeff Mercado
Nov 22 '18 at 7:48
@Eddie: semicolons are used for separating parameters to function calls. The way you're using it there, it should be pipes and it would work like you would expect.
– Jeff Mercado
Nov 22 '18 at 7:48
add a comment |
Another approach, which illustrates the power of jq quite nicely, is to use a filter named "sum" defined as follows:
def sum(f): reduce . as $row (0; . + ($row|f) );
To solve the particular problem at hand, one could then use the -s (--slurp) option as mentioned above, together with the expression:
{"a": sum(.a), "b": sum(.b) } # (2)
The expression labeled (2) only computes the two specified sums, but it is easy to generalize, e.g. as follows:
# Produce an object with the same keys as the first object in the
# input array, but with values equal to the sum of the corresponding
# values in all the objects.
def sumByKey:
. as $in
| reduce (.[0] | keys) as $key
( {}; . + {($key): ($in | sum(.[$key]))})
;
answered Feb 16 '15 at 6:08
peakpeak
34.7k94461
add a comment |
Another approach, which illustrates the power of jq quite nicely, is to use a filter named "sum" defined as follows:
def sum(f): reduce . as $row (0; . + ($row|f) );
To solve the particular problem at hand, one could then use the -s (--slurp) option as mentioned above, together with the expression:
{"a": sum(.a), "b": sum(.b) } # (2)
The expression labeled (2) only computes the two specified sums, but it is easy to generalize, e.g. as follows:
# Produce an object with the same keys as the first object in the
# input array, but with values equal to the sum of the corresponding
# values in all the objects.
def sumByKey:
. as $in
| reduce (.[0] | keys) as $key
( {}; . + {($key): ($in | sum(.[$key]))})
;
answered Feb 16 '15 at 6:08
peakpeak
34.7k94461
add a comment |
Another approach, which illustrates the power of jq quite nicely, is to use a filter named "sum" defined as follows:
def sum(f): reduce . as $row (0; . + ($row|f) );
To solve the particular problem at hand, one could then use the -s (--slurp) option as mentioned above, together with the expression:
{"a": sum(.a), "b": sum(.b) } # (2)
The expression labeled (2) only computes the two specified sums, but it is easy to generalize, e.g. as follows:
# Produce an object with the same keys as the first object in the
# input array, but with values equal to the sum of the corresponding
# values in all the objects.
def sumByKey:
. as $in
| reduce (.[0] | keys) as $key
( {}; . + {($key): ($in | sum(.[$key]))})
;
answered Feb 16 '15 at 6:08
peakpeak
34.7k94461
Another approach, which illustrates the power of jq quite nicely, is to use a filter named "sum" defined as follows:
def sum(f): reduce . as $row (0; . + ($row|f) );
To solve the particular problem at hand, one could then use the -s (--slurp) option as mentioned above, together with the expression:
{"a": sum(.a), "b": sum(.b) } # (2)
The expression labeled (2) only computes the two specified sums, but it is easy to generalize, e.g. as follows:
# Produce an object with the same keys as the first object in the
# input array, but with values equal to the sum of the corresponding
# values in all the objects.
def sumByKey:
. as $in
| reduce (.[0] | keys) as $key
( {}; . + {($key): ($in | sum(.[$key]))})
;
answered Feb 16 '15 at 6:08
peakpeak
34.7k94461
edited Nov 24 '17 at 18:23
answered Feb 16 '15 at 6:08
peakpeak
34.7k94461
answered Feb 16 '15 at 6:08
peakpeak
34.7k94461
answered Feb 16 '15 at 6:08
peakpeak
34.7k94461
34.7k94461
add a comment |
add a comment |
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f28484534%2fhow-do-i-sum-the-values-in-an-array-of-maps-in-jq%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
