array sliding with spark dataframes

up vote
-3
down vote

favorite

suppose the data is T_32_P_1_A_420_H_60_R_0.30841494477846165_S_0
using scala spark dataframe, how to split into the following format

T 32

P 1

A 420

H 60

R 0.30841494477846165

S 0

Any suggestions are greatly appreciated.

Thanks in advance

asked Nov 8 at 13:41

Shakti

You want a row of data with columns T, P, A, H, R, S?
– Terry Dactyl
Nov 8 at 14:17

Is the data ` T_32_P_1_A_420_H_60_R_0.30841494477846165_S_0 ` single row ? or the whole data ?
– Shankar Koirala
Nov 8 at 14:18

Can you add few more inputs and expected results
– Shankar Koirala
Nov 8 at 14:23

another example
– Shakti
Nov 9 at 5:06

add a comment |

up vote
-3
down vote

favorite

suppose the data is T_32_P_1_A_420_H_60_R_0.30841494477846165_S_0
using scala spark dataframe, how to split into the following format

T 32

P 1

A 420

H 60

R 0.30841494477846165

S 0

Any suggestions are greatly appreciated.

Thanks in advance

asked Nov 8 at 13:41

Shakti

You want a row of data with columns T, P, A, H, R, S?
– Terry Dactyl
Nov 8 at 14:17

Is the data ` T_32_P_1_A_420_H_60_R_0.30841494477846165_S_0 ` single row ? or the whole data ?
– Shankar Koirala
Nov 8 at 14:18

Can you add few more inputs and expected results
– Shankar Koirala
Nov 8 at 14:23

another example
– Shakti
Nov 9 at 5:06

add a comment |

up vote
-3
down vote

favorite

suppose the data is T_32_P_1_A_420_H_60_R_0.30841494477846165_S_0
using scala spark dataframe, how to split into the following format

T 32

P 1

A 420

H 60

R 0.30841494477846165

S 0

Any suggestions are greatly appreciated.

Thanks in advance

asked Nov 8 at 13:41

Shakti

suppose the data is T_32_P_1_A_420_H_60_R_0.30841494477846165_S_0
using scala spark dataframe, how to split into the following format

T 32

P 1

A 420

H 60

R 0.30841494477846165

S 0

Any suggestions are greatly appreciated.

Thanks in advance

apache-spark dataframe sliding

asked Nov 8 at 13:41

Shakti

asked Nov 8 at 13:41

Shakti

asked Nov 8 at 13:41

Shakti

asked Nov 8 at 13:41

Shakti

asked Nov 8 at 13:41

Shakti

You want a row of data with columns T, P, A, H, R, S?
– Terry Dactyl
Nov 8 at 14:17

Is the data ` T_32_P_1_A_420_H_60_R_0.30841494477846165_S_0 ` single row ? or the whole data ?
– Shankar Koirala
Nov 8 at 14:18

Can you add few more inputs and expected results
– Shankar Koirala
Nov 8 at 14:23

another example
– Shakti
Nov 9 at 5:06

add a comment |

You want a row of data with columns T, P, A, H, R, S?
– Terry Dactyl
Nov 8 at 14:17

Is the data ` T_32_P_1_A_420_H_60_R_0.30841494477846165_S_0 ` single row ? or the whole data ?
– Shankar Koirala
Nov 8 at 14:18

Can you add few more inputs and expected results
– Shankar Koirala
Nov 8 at 14:23

another example
– Shakti
Nov 9 at 5:06

You want a row of data with columns T, P, A, H, R, S?
– Terry Dactyl
Nov 8 at 14:17

Is the data ` T_32_P_1_A_420_H_60_R_0.30841494477846165_S_0 ` single row ? or the whole data ?
– Shankar Koirala
Nov 8 at 14:18

Can you add few more inputs and expected results
– Shankar Koirala
Nov 8 at 14:23

another example
– Shakti
Nov 9 at 5:06

add a comment |

1 Answer
1

active

oldest

votes

up vote
0
down vote

another example

+-------+-------------+-----------------------------------------------------------------------------+

|Pcode  |Pname        |Pdetails                                                                     |

+-------+-------------+-----------------------------------------------------------------------------+

|Water12|HimalayaWater|Price,1.20;Qty,250ml;Brand,Himalaya;Class,Liquid                             |

|Snack23|Mad Pringles |Price,0.65;Qty,165 g;Brand,MadLtd;Class,Snacks;Batch,12312334;Exp,12/Feb/2012|

+-------+-------------+-----------------------------------------------------------------------------+

i want pdeatils to be split into two columns Type and Value, and expected output is

+-------+-------------+-----+-----------+

|Pcode  |Pname        |Type |Value      |

+-------+-------------+-----+-----------+

|Water12|HimalayaWater|Price|1.20       |

|Water12|HimalayaWater|Qty  |250ml      |

|Water12|HimalayaWater|Brand|Himalaya   |

|Water12|HimalayaWater|Class|Liquid     |

|Snack23|Mad Pringles |Price|0.65       |

|Snack23|Mad Pringles |Qty  |165 g      |

|Snack23|Mad Pringles |Brand|MadLtd     |

|Snack23|Mad Pringles |Class|Snacks     |

|Snack23|Mad Pringles |Batch|12312334   |

|Snack23|Mad Pringles |Exp  |12/Feb/2012|

+-------+-------------+-----+-----------+

edited Nov 9 at 5:22

answered Nov 9 at 5:13

Shakti

add a comment |

Your Answer

StackExchange.ifUsing("editor", function () {
StackExchange.using("externalEditor", function () {
StackExchange.using("snippets", function () {
StackExchange.snippets.init();
});
});
}, "code-snippets");

StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "1"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);

StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});

function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});

}
});

draft saved

draft discarded

Sign up or log in

StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});

Post as a guest

Name

Required, but never shown

StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53208960%2farray-sliding-with-spark-dataframes%23new-answer', 'question_page');
}
);

Post as a guest

Name

Required, but never shown

1 Answer
1

active

oldest

votes

1 Answer
1

active

oldest

votes

up vote
0
down vote

another example

+-------+-------------+-----------------------------------------------------------------------------+

|Pcode  |Pname        |Pdetails                                                                     |

+-------+-------------+-----------------------------------------------------------------------------+

|Water12|HimalayaWater|Price,1.20;Qty,250ml;Brand,Himalaya;Class,Liquid                             |

|Snack23|Mad Pringles |Price,0.65;Qty,165 g;Brand,MadLtd;Class,Snacks;Batch,12312334;Exp,12/Feb/2012|

+-------+-------------+-----------------------------------------------------------------------------+

i want pdeatils to be split into two columns Type and Value, and expected output is

+-------+-------------+-----+-----------+

|Pcode  |Pname        |Type |Value      |

+-------+-------------+-----+-----------+

|Water12|HimalayaWater|Price|1.20       |

|Water12|HimalayaWater|Qty  |250ml      |

|Water12|HimalayaWater|Brand|Himalaya   |

|Water12|HimalayaWater|Class|Liquid     |

|Snack23|Mad Pringles |Price|0.65       |

|Snack23|Mad Pringles |Qty  |165 g      |

|Snack23|Mad Pringles |Brand|MadLtd     |

|Snack23|Mad Pringles |Class|Snacks     |

|Snack23|Mad Pringles |Batch|12312334   |

|Snack23|Mad Pringles |Exp  |12/Feb/2012|

+-------+-------------+-----+-----------+

edited Nov 9 at 5:22

answered Nov 9 at 5:13

Shakti

add a comment |

up vote
0
down vote

another example

+-------+-------------+-----------------------------------------------------------------------------+

|Pcode  |Pname        |Pdetails                                                                     |

+-------+-------------+-----------------------------------------------------------------------------+

|Water12|HimalayaWater|Price,1.20;Qty,250ml;Brand,Himalaya;Class,Liquid                             |

|Snack23|Mad Pringles |Price,0.65;Qty,165 g;Brand,MadLtd;Class,Snacks;Batch,12312334;Exp,12/Feb/2012|

+-------+-------------+-----------------------------------------------------------------------------+

i want pdeatils to be split into two columns Type and Value, and expected output is

+-------+-------------+-----+-----------+

|Pcode  |Pname        |Type |Value      |

+-------+-------------+-----+-----------+

|Water12|HimalayaWater|Price|1.20       |

|Water12|HimalayaWater|Qty  |250ml      |

|Water12|HimalayaWater|Brand|Himalaya   |

|Water12|HimalayaWater|Class|Liquid     |

|Snack23|Mad Pringles |Price|0.65       |

|Snack23|Mad Pringles |Qty  |165 g      |

|Snack23|Mad Pringles |Brand|MadLtd     |

|Snack23|Mad Pringles |Class|Snacks     |

|Snack23|Mad Pringles |Batch|12312334   |

|Snack23|Mad Pringles |Exp  |12/Feb/2012|

+-------+-------------+-----+-----------+

edited Nov 9 at 5:22

answered Nov 9 at 5:13

Shakti

add a comment |

up vote
0
down vote

another example

+-------+-------------+-----------------------------------------------------------------------------+

|Pcode  |Pname        |Pdetails                                                                     |

+-------+-------------+-----------------------------------------------------------------------------+

|Water12|HimalayaWater|Price,1.20;Qty,250ml;Brand,Himalaya;Class,Liquid                             |

|Snack23|Mad Pringles |Price,0.65;Qty,165 g;Brand,MadLtd;Class,Snacks;Batch,12312334;Exp,12/Feb/2012|

+-------+-------------+-----------------------------------------------------------------------------+

i want pdeatils to be split into two columns Type and Value, and expected output is

+-------+-------------+-----+-----------+

|Pcode  |Pname        |Type |Value      |

+-------+-------------+-----+-----------+

|Water12|HimalayaWater|Price|1.20       |

|Water12|HimalayaWater|Qty  |250ml      |

|Water12|HimalayaWater|Brand|Himalaya   |

|Water12|HimalayaWater|Class|Liquid     |

|Snack23|Mad Pringles |Price|0.65       |

|Snack23|Mad Pringles |Qty  |165 g      |

|Snack23|Mad Pringles |Brand|MadLtd     |

|Snack23|Mad Pringles |Class|Snacks     |

|Snack23|Mad Pringles |Batch|12312334   |

|Snack23|Mad Pringles |Exp  |12/Feb/2012|

+-------+-------------+-----+-----------+

edited Nov 9 at 5:22

answered Nov 9 at 5:13

Shakti

another example

+-------+-------------+-----------------------------------------------------------------------------+

|Pcode  |Pname        |Pdetails                                                                     |

+-------+-------------+-----------------------------------------------------------------------------+

|Water12|HimalayaWater|Price,1.20;Qty,250ml;Brand,Himalaya;Class,Liquid                             |

|Snack23|Mad Pringles |Price,0.65;Qty,165 g;Brand,MadLtd;Class,Snacks;Batch,12312334;Exp,12/Feb/2012|

+-------+-------------+-----------------------------------------------------------------------------+

i want pdeatils to be split into two columns Type and Value, and expected output is

+-------+-------------+-----+-----------+

|Pcode  |Pname        |Type |Value      |

+-------+-------------+-----+-----------+

|Water12|HimalayaWater|Price|1.20       |

|Water12|HimalayaWater|Qty  |250ml      |

|Water12|HimalayaWater|Brand|Himalaya   |

|Water12|HimalayaWater|Class|Liquid     |

|Snack23|Mad Pringles |Price|0.65       |

|Snack23|Mad Pringles |Qty  |165 g      |

|Snack23|Mad Pringles |Brand|MadLtd     |

|Snack23|Mad Pringles |Class|Snacks     |

|Snack23|Mad Pringles |Batch|12312334   |

|Snack23|Mad Pringles |Exp  |12/Feb/2012|

+-------+-------------+-----+-----------+

edited Nov 9 at 5:22

answered Nov 9 at 5:13

Shakti

edited Nov 9 at 5:22

answered Nov 9 at 5:13

Shakti

answered Nov 9 at 5:13

Shakti

answered Nov 9 at 5:13

Shakti

add a comment |

draft saved

draft discarded

draft saved

draft discarded

Sign up or log in

StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});

Post as a guest

Name

Required, but never shown

Post as a guest

Name

Required, but never shown

Sign up or log in

StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});

Post as a guest

Name

Required, but never shown

Sign up or log in

StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});

Post as a guest

Name

Required, but never shown

Sign up or log in

StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});

Post as a guest

Name

Required, but never shown

Name

Required, but never shown

Name

Required, but never shown

This page is only for reference, If you need detailed information, please check here

搜尋此網誌

Agfdhyk