Curl works fine but Perl LWP failed to post Unicode code to HTTPS server











up vote
0
down vote

favorite












My snippet as below, using curl I can send the Unicode string to the server
and after push notify the message to my mobile phone, displayed correctly the Chinese characters on the phone.



      curl  "https://oapi.dingtalk.com/robot/send?access_token=$ACCESS_TOKEN" 
-H 'Content-Type: application/json'
-d '{"msgtype": "text", "text": {
"content": "u6d4bu8bd5",
}}'


But when I use the Perl LWP library to send the same content to the server,
I only got 6d4b8bd5 displayed on the app.
I've tried to use tcpdump to compare the difference between curl and LWP sent out buffer, but as it's https server it's not working.



    #!/usr/bin/perl -w
use strict;
use warnings;
use Cpanel::JSON::XS qw( encode_json );
# Create a user agent object
use LWP::UserAgent;
$ENV{PERL_LWP_SSL_VERIFY_HOSTNAME} = 0;

my $ua = LWP::UserAgent->new;
$ua->agent("curl/7.47.0");


# Create a request
my $req = HTTP::Request->new(POST => 'https://oapi.dingtalk.com/robot/send?access_token=bf47148ea6a1b125395f9313873ac508');

$req->content_type('application/json;charset=utf-8');

my $var='u6d4bu8bd5';

my $message = encode_json({ msgtype => "text", text => { content => $var } });

$req->content($message);

# Pass request to the user agent and get a response back
my $res = $ua->request($req);

# Check the outcome of the response
if ($res->is_success) {
print $res->content;
} else {
print $res->status_line, "n";
}


any suggestions? Thanks!










share|improve this question




























    up vote
    0
    down vote

    favorite












    My snippet as below, using curl I can send the Unicode string to the server
    and after push notify the message to my mobile phone, displayed correctly the Chinese characters on the phone.



          curl  "https://oapi.dingtalk.com/robot/send?access_token=$ACCESS_TOKEN" 
    -H 'Content-Type: application/json'
    -d '{"msgtype": "text", "text": {
    "content": "u6d4bu8bd5",
    }}'


    But when I use the Perl LWP library to send the same content to the server,
    I only got 6d4b8bd5 displayed on the app.
    I've tried to use tcpdump to compare the difference between curl and LWP sent out buffer, but as it's https server it's not working.



        #!/usr/bin/perl -w
    use strict;
    use warnings;
    use Cpanel::JSON::XS qw( encode_json );
    # Create a user agent object
    use LWP::UserAgent;
    $ENV{PERL_LWP_SSL_VERIFY_HOSTNAME} = 0;

    my $ua = LWP::UserAgent->new;
    $ua->agent("curl/7.47.0");


    # Create a request
    my $req = HTTP::Request->new(POST => 'https://oapi.dingtalk.com/robot/send?access_token=bf47148ea6a1b125395f9313873ac508');

    $req->content_type('application/json;charset=utf-8');

    my $var='u6d4bu8bd5';

    my $message = encode_json({ msgtype => "text", text => { content => $var } });

    $req->content($message);

    # Pass request to the user agent and get a response back
    my $res = $ua->request($req);

    # Check the outcome of the response
    if ($res->is_success) {
    print $res->content;
    } else {
    print $res->status_line, "n";
    }


    any suggestions? Thanks!










    share|improve this question


























      up vote
      0
      down vote

      favorite









      up vote
      0
      down vote

      favorite











      My snippet as below, using curl I can send the Unicode string to the server
      and after push notify the message to my mobile phone, displayed correctly the Chinese characters on the phone.



            curl  "https://oapi.dingtalk.com/robot/send?access_token=$ACCESS_TOKEN" 
      -H 'Content-Type: application/json'
      -d '{"msgtype": "text", "text": {
      "content": "u6d4bu8bd5",
      }}'


      But when I use the Perl LWP library to send the same content to the server,
      I only got 6d4b8bd5 displayed on the app.
      I've tried to use tcpdump to compare the difference between curl and LWP sent out buffer, but as it's https server it's not working.



          #!/usr/bin/perl -w
      use strict;
      use warnings;
      use Cpanel::JSON::XS qw( encode_json );
      # Create a user agent object
      use LWP::UserAgent;
      $ENV{PERL_LWP_SSL_VERIFY_HOSTNAME} = 0;

      my $ua = LWP::UserAgent->new;
      $ua->agent("curl/7.47.0");


      # Create a request
      my $req = HTTP::Request->new(POST => 'https://oapi.dingtalk.com/robot/send?access_token=bf47148ea6a1b125395f9313873ac508');

      $req->content_type('application/json;charset=utf-8');

      my $var='u6d4bu8bd5';

      my $message = encode_json({ msgtype => "text", text => { content => $var } });

      $req->content($message);

      # Pass request to the user agent and get a response back
      my $res = $ua->request($req);

      # Check the outcome of the response
      if ($res->is_success) {
      print $res->content;
      } else {
      print $res->status_line, "n";
      }


      any suggestions? Thanks!










      share|improve this question















      My snippet as below, using curl I can send the Unicode string to the server
      and after push notify the message to my mobile phone, displayed correctly the Chinese characters on the phone.



            curl  "https://oapi.dingtalk.com/robot/send?access_token=$ACCESS_TOKEN" 
      -H 'Content-Type: application/json'
      -d '{"msgtype": "text", "text": {
      "content": "u6d4bu8bd5",
      }}'


      But when I use the Perl LWP library to send the same content to the server,
      I only got 6d4b8bd5 displayed on the app.
      I've tried to use tcpdump to compare the difference between curl and LWP sent out buffer, but as it's https server it's not working.



          #!/usr/bin/perl -w
      use strict;
      use warnings;
      use Cpanel::JSON::XS qw( encode_json );
      # Create a user agent object
      use LWP::UserAgent;
      $ENV{PERL_LWP_SSL_VERIFY_HOSTNAME} = 0;

      my $ua = LWP::UserAgent->new;
      $ua->agent("curl/7.47.0");


      # Create a request
      my $req = HTTP::Request->new(POST => 'https://oapi.dingtalk.com/robot/send?access_token=bf47148ea6a1b125395f9313873ac508');

      $req->content_type('application/json;charset=utf-8');

      my $var='u6d4bu8bd5';

      my $message = encode_json({ msgtype => "text", text => { content => $var } });

      $req->content($message);

      # Pass request to the user agent and get a response back
      my $res = $ua->request($req);

      # Check the outcome of the response
      if ($res->is_success) {
      print $res->content;
      } else {
      print $res->status_line, "n";
      }


      any suggestions? Thanks!







      perl curl unicode https lwp






      share|improve this question















      share|improve this question













      share|improve this question




      share|improve this question








      edited Nov 12 at 8:00

























      asked Nov 12 at 6:18









      codebyte

      52




      52
























          1 Answer
          1






          active

          oldest

          votes

















          up vote
          2
          down vote



          accepted










          Earlier, you didn't encode. Now, you double encode. encode_json is already handling the encoding to JSON, so don't do it manually too!



          If you provide the 12 characters u6d4bu8bd5, it will receive the 12 characters u6d4bu8bd5.



          If you provide the two characters U+6D4B and U+8BD5, it will receive the two characters U+6D4B and U+8BD5.




          • my $var = chr(0x6D4B) . chr(0x8BD5);

          • my $var = "x{6D4B}x{8BD5}";

          • my $var = "N{U+6D4B}N{U+8BD5}";

          • use utf8; my $var = "测试";

          • etc






          share|improve this answer





















          • Thanks! So it means that encode_json api cannot support using the u6d4bu8bd5 format and must use the format you mentioned above. If I want to stick to u6d4bu8bd5, then I have to find another way to pass the variable in the inner json value.
            – codebyte
            Nov 12 at 8:48










          • Any suggestion to work with u6d4bu8bd5 string format as it's the database format ? use --trace-ascii /dev/stdout with curl I can see that curl is sending out "content": "u6d4bu8bd5" to the https server.
            – codebyte
            Nov 12 at 9:52










          • If you checked the output of your code (with my change), you'd also see u6d4bu8bd5 within the output. Again, let encode_json handle the JSON encoding!
            – ikegami
            Nov 12 at 14:22












          • If you have u6d4bu8bd5 in the database, you've done something wrong. You shouldn't see u6d4bu8bd5 outside of JSON. Maybe you did some shoddy manual parsing instead of using decode_json or similar? (Of course, if the database field contains proper JSON, then just decode_json or similar as the first step of getting the value you want.)
            – ikegami
            Nov 12 at 14:27












          • u6d4bu8bd5 is from another REST api, for example, 182383:u6d4bu8bd5,182384:u7a4bu2bd5,etc, the list is available from the website. Then I save the list in local JSON array and use the number key to find its value and send it to my backend server, which will push to the registered app.
            – codebyte
            Nov 13 at 3:39













          Your Answer






          StackExchange.ifUsing("editor", function () {
          StackExchange.using("externalEditor", function () {
          StackExchange.using("snippets", function () {
          StackExchange.snippets.init();
          });
          });
          }, "code-snippets");

          StackExchange.ready(function() {
          var channelOptions = {
          tags: "".split(" "),
          id: "1"
          };
          initTagRenderer("".split(" "), "".split(" "), channelOptions);

          StackExchange.using("externalEditor", function() {
          // Have to fire editor after snippets, if snippets enabled
          if (StackExchange.settings.snippets.snippetsEnabled) {
          StackExchange.using("snippets", function() {
          createEditor();
          });
          }
          else {
          createEditor();
          }
          });

          function createEditor() {
          StackExchange.prepareEditor({
          heartbeatType: 'answer',
          convertImagesToLinks: true,
          noModals: true,
          showLowRepImageUploadWarning: true,
          reputationToPostImages: 10,
          bindNavPrevention: true,
          postfix: "",
          imageUploader: {
          brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
          contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
          allowUrls: true
          },
          onDemand: true,
          discardSelector: ".discard-answer"
          ,immediatelyShowMarkdownHelp:true
          });


          }
          });














          draft saved

          draft discarded


















          StackExchange.ready(
          function () {
          StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53256796%2fcurl-works-fine-but-perl-lwp-failed-to-post-unicode-code-to-https-server%23new-answer', 'question_page');
          }
          );

          Post as a guest















          Required, but never shown

























          1 Answer
          1






          active

          oldest

          votes








          1 Answer
          1






          active

          oldest

          votes









          active

          oldest

          votes






          active

          oldest

          votes








          up vote
          2
          down vote



          accepted










          Earlier, you didn't encode. Now, you double encode. encode_json is already handling the encoding to JSON, so don't do it manually too!



          If you provide the 12 characters u6d4bu8bd5, it will receive the 12 characters u6d4bu8bd5.



          If you provide the two characters U+6D4B and U+8BD5, it will receive the two characters U+6D4B and U+8BD5.




          • my $var = chr(0x6D4B) . chr(0x8BD5);

          • my $var = "x{6D4B}x{8BD5}";

          • my $var = "N{U+6D4B}N{U+8BD5}";

          • use utf8; my $var = "测试";

          • etc






          share|improve this answer





















          • Thanks! So it means that encode_json api cannot support using the u6d4bu8bd5 format and must use the format you mentioned above. If I want to stick to u6d4bu8bd5, then I have to find another way to pass the variable in the inner json value.
            – codebyte
            Nov 12 at 8:48










          • Any suggestion to work with u6d4bu8bd5 string format as it's the database format ? use --trace-ascii /dev/stdout with curl I can see that curl is sending out "content": "u6d4bu8bd5" to the https server.
            – codebyte
            Nov 12 at 9:52










          • If you checked the output of your code (with my change), you'd also see u6d4bu8bd5 within the output. Again, let encode_json handle the JSON encoding!
            – ikegami
            Nov 12 at 14:22












          • If you have u6d4bu8bd5 in the database, you've done something wrong. You shouldn't see u6d4bu8bd5 outside of JSON. Maybe you did some shoddy manual parsing instead of using decode_json or similar? (Of course, if the database field contains proper JSON, then just decode_json or similar as the first step of getting the value you want.)
            – ikegami
            Nov 12 at 14:27












          • u6d4bu8bd5 is from another REST api, for example, 182383:u6d4bu8bd5,182384:u7a4bu2bd5,etc, the list is available from the website. Then I save the list in local JSON array and use the number key to find its value and send it to my backend server, which will push to the registered app.
            – codebyte
            Nov 13 at 3:39

















          up vote
          2
          down vote



          accepted










          Earlier, you didn't encode. Now, you double encode. encode_json is already handling the encoding to JSON, so don't do it manually too!



          If you provide the 12 characters u6d4bu8bd5, it will receive the 12 characters u6d4bu8bd5.



          If you provide the two characters U+6D4B and U+8BD5, it will receive the two characters U+6D4B and U+8BD5.




          • my $var = chr(0x6D4B) . chr(0x8BD5);

          • my $var = "x{6D4B}x{8BD5}";

          • my $var = "N{U+6D4B}N{U+8BD5}";

          • use utf8; my $var = "测试";

          • etc






          share|improve this answer





















          • Thanks! So it means that encode_json api cannot support using the u6d4bu8bd5 format and must use the format you mentioned above. If I want to stick to u6d4bu8bd5, then I have to find another way to pass the variable in the inner json value.
            – codebyte
            Nov 12 at 8:48










          • Any suggestion to work with u6d4bu8bd5 string format as it's the database format ? use --trace-ascii /dev/stdout with curl I can see that curl is sending out "content": "u6d4bu8bd5" to the https server.
            – codebyte
            Nov 12 at 9:52










          • If you checked the output of your code (with my change), you'd also see u6d4bu8bd5 within the output. Again, let encode_json handle the JSON encoding!
            – ikegami
            Nov 12 at 14:22












          • If you have u6d4bu8bd5 in the database, you've done something wrong. You shouldn't see u6d4bu8bd5 outside of JSON. Maybe you did some shoddy manual parsing instead of using decode_json or similar? (Of course, if the database field contains proper JSON, then just decode_json or similar as the first step of getting the value you want.)
            – ikegami
            Nov 12 at 14:27












          • u6d4bu8bd5 is from another REST api, for example, 182383:u6d4bu8bd5,182384:u7a4bu2bd5,etc, the list is available from the website. Then I save the list in local JSON array and use the number key to find its value and send it to my backend server, which will push to the registered app.
            – codebyte
            Nov 13 at 3:39















          up vote
          2
          down vote



          accepted







          up vote
          2
          down vote



          accepted






          Earlier, you didn't encode. Now, you double encode. encode_json is already handling the encoding to JSON, so don't do it manually too!



          If you provide the 12 characters u6d4bu8bd5, it will receive the 12 characters u6d4bu8bd5.



          If you provide the two characters U+6D4B and U+8BD5, it will receive the two characters U+6D4B and U+8BD5.




          • my $var = chr(0x6D4B) . chr(0x8BD5);

          • my $var = "x{6D4B}x{8BD5}";

          • my $var = "N{U+6D4B}N{U+8BD5}";

          • use utf8; my $var = "测试";

          • etc






          share|improve this answer












          Earlier, you didn't encode. Now, you double encode. encode_json is already handling the encoding to JSON, so don't do it manually too!



          If you provide the 12 characters u6d4bu8bd5, it will receive the 12 characters u6d4bu8bd5.



          If you provide the two characters U+6D4B and U+8BD5, it will receive the two characters U+6D4B and U+8BD5.




          • my $var = chr(0x6D4B) . chr(0x8BD5);

          • my $var = "x{6D4B}x{8BD5}";

          • my $var = "N{U+6D4B}N{U+8BD5}";

          • use utf8; my $var = "测试";

          • etc







          share|improve this answer












          share|improve this answer



          share|improve this answer










          answered Nov 12 at 7:06









          ikegami

          260k11174394




          260k11174394












          • Thanks! So it means that encode_json api cannot support using the u6d4bu8bd5 format and must use the format you mentioned above. If I want to stick to u6d4bu8bd5, then I have to find another way to pass the variable in the inner json value.
            – codebyte
            Nov 12 at 8:48










          • Any suggestion to work with u6d4bu8bd5 string format as it's the database format ? use --trace-ascii /dev/stdout with curl I can see that curl is sending out "content": "u6d4bu8bd5" to the https server.
            – codebyte
            Nov 12 at 9:52










          • If you checked the output of your code (with my change), you'd also see u6d4bu8bd5 within the output. Again, let encode_json handle the JSON encoding!
            – ikegami
            Nov 12 at 14:22












          • If you have u6d4bu8bd5 in the database, you've done something wrong. You shouldn't see u6d4bu8bd5 outside of JSON. Maybe you did some shoddy manual parsing instead of using decode_json or similar? (Of course, if the database field contains proper JSON, then just decode_json or similar as the first step of getting the value you want.)
            – ikegami
            Nov 12 at 14:27












          • u6d4bu8bd5 is from another REST api, for example, 182383:u6d4bu8bd5,182384:u7a4bu2bd5,etc, the list is available from the website. Then I save the list in local JSON array and use the number key to find its value and send it to my backend server, which will push to the registered app.
            – codebyte
            Nov 13 at 3:39




















          • Thanks! So it means that encode_json api cannot support using the u6d4bu8bd5 format and must use the format you mentioned above. If I want to stick to u6d4bu8bd5, then I have to find another way to pass the variable in the inner json value.
            – codebyte
            Nov 12 at 8:48










          • Any suggestion to work with u6d4bu8bd5 string format as it's the database format ? use --trace-ascii /dev/stdout with curl I can see that curl is sending out "content": "u6d4bu8bd5" to the https server.
            – codebyte
            Nov 12 at 9:52










          • If you checked the output of your code (with my change), you'd also see u6d4bu8bd5 within the output. Again, let encode_json handle the JSON encoding!
            – ikegami
            Nov 12 at 14:22












          • If you have u6d4bu8bd5 in the database, you've done something wrong. You shouldn't see u6d4bu8bd5 outside of JSON. Maybe you did some shoddy manual parsing instead of using decode_json or similar? (Of course, if the database field contains proper JSON, then just decode_json or similar as the first step of getting the value you want.)
            – ikegami
            Nov 12 at 14:27












          • u6d4bu8bd5 is from another REST api, for example, 182383:u6d4bu8bd5,182384:u7a4bu2bd5,etc, the list is available from the website. Then I save the list in local JSON array and use the number key to find its value and send it to my backend server, which will push to the registered app.
            – codebyte
            Nov 13 at 3:39


















          Thanks! So it means that encode_json api cannot support using the u6d4bu8bd5 format and must use the format you mentioned above. If I want to stick to u6d4bu8bd5, then I have to find another way to pass the variable in the inner json value.
          – codebyte
          Nov 12 at 8:48




          Thanks! So it means that encode_json api cannot support using the u6d4bu8bd5 format and must use the format you mentioned above. If I want to stick to u6d4bu8bd5, then I have to find another way to pass the variable in the inner json value.
          – codebyte
          Nov 12 at 8:48












          Any suggestion to work with u6d4bu8bd5 string format as it's the database format ? use --trace-ascii /dev/stdout with curl I can see that curl is sending out "content": "u6d4bu8bd5" to the https server.
          – codebyte
          Nov 12 at 9:52




          Any suggestion to work with u6d4bu8bd5 string format as it's the database format ? use --trace-ascii /dev/stdout with curl I can see that curl is sending out "content": "u6d4bu8bd5" to the https server.
          – codebyte
          Nov 12 at 9:52












          If you checked the output of your code (with my change), you'd also see u6d4bu8bd5 within the output. Again, let encode_json handle the JSON encoding!
          – ikegami
          Nov 12 at 14:22






          If you checked the output of your code (with my change), you'd also see u6d4bu8bd5 within the output. Again, let encode_json handle the JSON encoding!
          – ikegami
          Nov 12 at 14:22














          If you have u6d4bu8bd5 in the database, you've done something wrong. You shouldn't see u6d4bu8bd5 outside of JSON. Maybe you did some shoddy manual parsing instead of using decode_json or similar? (Of course, if the database field contains proper JSON, then just decode_json or similar as the first step of getting the value you want.)
          – ikegami
          Nov 12 at 14:27






          If you have u6d4bu8bd5 in the database, you've done something wrong. You shouldn't see u6d4bu8bd5 outside of JSON. Maybe you did some shoddy manual parsing instead of using decode_json or similar? (Of course, if the database field contains proper JSON, then just decode_json or similar as the first step of getting the value you want.)
          – ikegami
          Nov 12 at 14:27














          u6d4bu8bd5 is from another REST api, for example, 182383:u6d4bu8bd5,182384:u7a4bu2bd5,etc, the list is available from the website. Then I save the list in local JSON array and use the number key to find its value and send it to my backend server, which will push to the registered app.
          – codebyte
          Nov 13 at 3:39






          u6d4bu8bd5 is from another REST api, for example, 182383:u6d4bu8bd5,182384:u7a4bu2bd5,etc, the list is available from the website. Then I save the list in local JSON array and use the number key to find its value and send it to my backend server, which will push to the registered app.
          – codebyte
          Nov 13 at 3:39




















          draft saved

          draft discarded




















































          Thanks for contributing an answer to Stack Overflow!


          • Please be sure to answer the question. Provide details and share your research!

          But avoid



          • Asking for help, clarification, or responding to other answers.

          • Making statements based on opinion; back them up with references or personal experience.


          To learn more, see our tips on writing great answers.





          Some of your past answers have not been well-received, and you're in danger of being blocked from answering.


          Please pay close attention to the following guidance:


          • Please be sure to answer the question. Provide details and share your research!

          But avoid



          • Asking for help, clarification, or responding to other answers.

          • Making statements based on opinion; back them up with references or personal experience.


          To learn more, see our tips on writing great answers.




          draft saved


          draft discarded














          StackExchange.ready(
          function () {
          StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53256796%2fcurl-works-fine-but-perl-lwp-failed-to-post-unicode-code-to-https-server%23new-answer', 'question_page');
          }
          );

          Post as a guest















          Required, but never shown





















































          Required, but never shown














          Required, but never shown












          Required, but never shown







          Required, but never shown

































          Required, but never shown














          Required, but never shown












          Required, but never shown







          Required, but never shown







          Popular posts from this blog

          Guess what letter conforming each word

          Port of Spain

          Run scheduled task as local user group (not BUILTIN)