Acoustic Echo Cancellation (AEC) on Xamarin.iOS












2















I am currently working on cross platform Voip app development based on Xamarin.iOS platform. I did search about the AEC implementation on iOS but most topics are related to Objective-C.
What I have already achieved now is: I could use Audiotoolbox (audio queue) to obtain the input raw sound data from mic and send it out with socket. But during testing with other device, I am experiencing very clear echo on the phone. Here is code:



private void SetupInputQueue()
{
inputQueue = new InputAudioQueue(audioStreamBasicDescription);
//sendbuffer initialization
heading = Encoding.ASCII.GetBytes("msg ");
sendBuffer = new byte[CountAudioBuffers];
for (int i = 0; i < CountAudioBuffers; i++)
{
sendBuffer[i] = new byte[516];
for (int j = 0; j < heading.Length; j++)
{
sendBuffer[i][j] = heading[j];
}
}

for (int count = 0; count < CountAudioBuffers; count++)
{
IntPtr bufferpointer;
inputQueue.AllocateBuffer(AudioBufferLength, out bufferpointer);
inputQueue.EnqueueBuffer(bufferpointer, AudioBufferLength, null);
}
inputQueue.InputCompleted += HandleInputCompleted;
}

private void HandleInputCompleted(object sender, InputCompletedEventArgs e)
{
unsafe
{
byte* shortPtr = (byte*)e.UnsafeBuffer->AudioData;
for (int count = heading.Length; count < sendBuffer[sendOutIndex].Length; count++)
{
sendBuffer[sendOutIndex][count] = *shortPtr;
shortPtr++;
}
}
socket.SendTo(sendBuffer[sendOutIndex], master);
this.inputQueue.EnqueueBuffer(e.IntPtrBuffer, AudioBufferLength, null);
sendOutIndex = (sendOutIndex + 1) % CountAudioBuffers;
}


According to AEC on OSX using AudioQueue, I learnt the hint that I should pass the raw sound data to I/O unit (audio unit?). But due to the lack of examples in Xamarin.iOS(c#), I can't figure out how to implement this process in detail. Could any one familiar with Voip app development on Xamarin platform give me some example to study? Very appreciate any help or hint on this.



(Nov, 21, 2018)Some related posts I found:Record audio with audio unit Audio unit Callbacks An audio unit example










share|improve this question





























    2















    I am currently working on cross platform Voip app development based on Xamarin.iOS platform. I did search about the AEC implementation on iOS but most topics are related to Objective-C.
    What I have already achieved now is: I could use Audiotoolbox (audio queue) to obtain the input raw sound data from mic and send it out with socket. But during testing with other device, I am experiencing very clear echo on the phone. Here is code:



    private void SetupInputQueue()
    {
    inputQueue = new InputAudioQueue(audioStreamBasicDescription);
    //sendbuffer initialization
    heading = Encoding.ASCII.GetBytes("msg ");
    sendBuffer = new byte[CountAudioBuffers];
    for (int i = 0; i < CountAudioBuffers; i++)
    {
    sendBuffer[i] = new byte[516];
    for (int j = 0; j < heading.Length; j++)
    {
    sendBuffer[i][j] = heading[j];
    }
    }

    for (int count = 0; count < CountAudioBuffers; count++)
    {
    IntPtr bufferpointer;
    inputQueue.AllocateBuffer(AudioBufferLength, out bufferpointer);
    inputQueue.EnqueueBuffer(bufferpointer, AudioBufferLength, null);
    }
    inputQueue.InputCompleted += HandleInputCompleted;
    }

    private void HandleInputCompleted(object sender, InputCompletedEventArgs e)
    {
    unsafe
    {
    byte* shortPtr = (byte*)e.UnsafeBuffer->AudioData;
    for (int count = heading.Length; count < sendBuffer[sendOutIndex].Length; count++)
    {
    sendBuffer[sendOutIndex][count] = *shortPtr;
    shortPtr++;
    }
    }
    socket.SendTo(sendBuffer[sendOutIndex], master);
    this.inputQueue.EnqueueBuffer(e.IntPtrBuffer, AudioBufferLength, null);
    sendOutIndex = (sendOutIndex + 1) % CountAudioBuffers;
    }


    According to AEC on OSX using AudioQueue, I learnt the hint that I should pass the raw sound data to I/O unit (audio unit?). But due to the lack of examples in Xamarin.iOS(c#), I can't figure out how to implement this process in detail. Could any one familiar with Voip app development on Xamarin platform give me some example to study? Very appreciate any help or hint on this.



    (Nov, 21, 2018)Some related posts I found:Record audio with audio unit Audio unit Callbacks An audio unit example










    share|improve this question



























      2












      2








      2








      I am currently working on cross platform Voip app development based on Xamarin.iOS platform. I did search about the AEC implementation on iOS but most topics are related to Objective-C.
      What I have already achieved now is: I could use Audiotoolbox (audio queue) to obtain the input raw sound data from mic and send it out with socket. But during testing with other device, I am experiencing very clear echo on the phone. Here is code:



      private void SetupInputQueue()
      {
      inputQueue = new InputAudioQueue(audioStreamBasicDescription);
      //sendbuffer initialization
      heading = Encoding.ASCII.GetBytes("msg ");
      sendBuffer = new byte[CountAudioBuffers];
      for (int i = 0; i < CountAudioBuffers; i++)
      {
      sendBuffer[i] = new byte[516];
      for (int j = 0; j < heading.Length; j++)
      {
      sendBuffer[i][j] = heading[j];
      }
      }

      for (int count = 0; count < CountAudioBuffers; count++)
      {
      IntPtr bufferpointer;
      inputQueue.AllocateBuffer(AudioBufferLength, out bufferpointer);
      inputQueue.EnqueueBuffer(bufferpointer, AudioBufferLength, null);
      }
      inputQueue.InputCompleted += HandleInputCompleted;
      }

      private void HandleInputCompleted(object sender, InputCompletedEventArgs e)
      {
      unsafe
      {
      byte* shortPtr = (byte*)e.UnsafeBuffer->AudioData;
      for (int count = heading.Length; count < sendBuffer[sendOutIndex].Length; count++)
      {
      sendBuffer[sendOutIndex][count] = *shortPtr;
      shortPtr++;
      }
      }
      socket.SendTo(sendBuffer[sendOutIndex], master);
      this.inputQueue.EnqueueBuffer(e.IntPtrBuffer, AudioBufferLength, null);
      sendOutIndex = (sendOutIndex + 1) % CountAudioBuffers;
      }


      According to AEC on OSX using AudioQueue, I learnt the hint that I should pass the raw sound data to I/O unit (audio unit?). But due to the lack of examples in Xamarin.iOS(c#), I can't figure out how to implement this process in detail. Could any one familiar with Voip app development on Xamarin platform give me some example to study? Very appreciate any help or hint on this.



      (Nov, 21, 2018)Some related posts I found:Record audio with audio unit Audio unit Callbacks An audio unit example










      share|improve this question
















      I am currently working on cross platform Voip app development based on Xamarin.iOS platform. I did search about the AEC implementation on iOS but most topics are related to Objective-C.
      What I have already achieved now is: I could use Audiotoolbox (audio queue) to obtain the input raw sound data from mic and send it out with socket. But during testing with other device, I am experiencing very clear echo on the phone. Here is code:



      private void SetupInputQueue()
      {
      inputQueue = new InputAudioQueue(audioStreamBasicDescription);
      //sendbuffer initialization
      heading = Encoding.ASCII.GetBytes("msg ");
      sendBuffer = new byte[CountAudioBuffers];
      for (int i = 0; i < CountAudioBuffers; i++)
      {
      sendBuffer[i] = new byte[516];
      for (int j = 0; j < heading.Length; j++)
      {
      sendBuffer[i][j] = heading[j];
      }
      }

      for (int count = 0; count < CountAudioBuffers; count++)
      {
      IntPtr bufferpointer;
      inputQueue.AllocateBuffer(AudioBufferLength, out bufferpointer);
      inputQueue.EnqueueBuffer(bufferpointer, AudioBufferLength, null);
      }
      inputQueue.InputCompleted += HandleInputCompleted;
      }

      private void HandleInputCompleted(object sender, InputCompletedEventArgs e)
      {
      unsafe
      {
      byte* shortPtr = (byte*)e.UnsafeBuffer->AudioData;
      for (int count = heading.Length; count < sendBuffer[sendOutIndex].Length; count++)
      {
      sendBuffer[sendOutIndex][count] = *shortPtr;
      shortPtr++;
      }
      }
      socket.SendTo(sendBuffer[sendOutIndex], master);
      this.inputQueue.EnqueueBuffer(e.IntPtrBuffer, AudioBufferLength, null);
      sendOutIndex = (sendOutIndex + 1) % CountAudioBuffers;
      }


      According to AEC on OSX using AudioQueue, I learnt the hint that I should pass the raw sound data to I/O unit (audio unit?). But due to the lack of examples in Xamarin.iOS(c#), I can't figure out how to implement this process in detail. Could any one familiar with Voip app development on Xamarin platform give me some example to study? Very appreciate any help or hint on this.



      (Nov, 21, 2018)Some related posts I found:Record audio with audio unit Audio unit Callbacks An audio unit example







      c# xamarin.ios voip echo-cancellation






      share|improve this question















      share|improve this question













      share|improve this question




      share|improve this question








      edited Nov 21 '18 at 15:09







      Akatsuki821

















      asked Nov 20 '18 at 21:42









      Akatsuki821Akatsuki821

      1114




      1114
























          2 Answers
          2






          active

          oldest

          votes


















          0














          I suggest to analyze the nature of the echo you are having in your system (for example test the echo path). Perhaps it is outside the capabilities of your built-in echo cancellation.






          share|improve this answer
























          • Thanks for giving suggestions. We figured out the way to turn on system build-in AEC and it works as expected. The article you share are very interesting to read. Thanks

            – Akatsuki821
            Nov 28 '18 at 15:29



















          0














          Finally we figured out how to turn on the iOS build-in AEC on Xamarin.iOS platform.
          The documentation here helped a lot to have a better understanding of the details of Audio Unit (even though this document is for iOS native development environment) and the working mechanism. If you want to precisely control every audio unit and understand parameters in the functions, this document is a must read.



          The work here provide a very nice start point to do experiment on Audio Unit. I made some modifications in step 2 of this post. The key part is that we have to use AudioTypeOutput.VoiceProcessingIO, this turns on AEC.



          public void prepareAudioUnit()
          {

          var _audioComponent = AudioComponent.FindComponent(AudioTypeOutput.VoiceProcessingIO);

          audioUnit = new AudioUnit.AudioUnit(_audioComponent);

          audioUnit.SetEnableIO(true,
          AudioUnitScopeType.Input,
          1 // Remote Input
          );

          // setting audio format
          audioUnit.SetAudioFormat(audioStreamBasicDesc,
          AudioUnitScopeType.Output,
          1
          );

          audioUnit.SetInputCallback(input_CallBack, AudioUnitScopeType.Input, 1);

          audioUnit.SetRenderCallback(render_CallBack, AudioUnitScopeType.Global, 0);


          audioUnit.Initialize();
          audioUnit.Start();
          }





          share|improve this answer























            Your Answer






            StackExchange.ifUsing("editor", function () {
            StackExchange.using("externalEditor", function () {
            StackExchange.using("snippets", function () {
            StackExchange.snippets.init();
            });
            });
            }, "code-snippets");

            StackExchange.ready(function() {
            var channelOptions = {
            tags: "".split(" "),
            id: "1"
            };
            initTagRenderer("".split(" "), "".split(" "), channelOptions);

            StackExchange.using("externalEditor", function() {
            // Have to fire editor after snippets, if snippets enabled
            if (StackExchange.settings.snippets.snippetsEnabled) {
            StackExchange.using("snippets", function() {
            createEditor();
            });
            }
            else {
            createEditor();
            }
            });

            function createEditor() {
            StackExchange.prepareEditor({
            heartbeatType: 'answer',
            autoActivateHeartbeat: false,
            convertImagesToLinks: true,
            noModals: true,
            showLowRepImageUploadWarning: true,
            reputationToPostImages: 10,
            bindNavPrevention: true,
            postfix: "",
            imageUploader: {
            brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
            contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
            allowUrls: true
            },
            onDemand: true,
            discardSelector: ".discard-answer"
            ,immediatelyShowMarkdownHelp:true
            });


            }
            });














            draft saved

            draft discarded


















            StackExchange.ready(
            function () {
            StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53402009%2facoustic-echo-cancellation-aec-on-xamarin-ios%23new-answer', 'question_page');
            }
            );

            Post as a guest















            Required, but never shown

























            2 Answers
            2






            active

            oldest

            votes








            2 Answers
            2






            active

            oldest

            votes









            active

            oldest

            votes






            active

            oldest

            votes









            0














            I suggest to analyze the nature of the echo you are having in your system (for example test the echo path). Perhaps it is outside the capabilities of your built-in echo cancellation.






            share|improve this answer
























            • Thanks for giving suggestions. We figured out the way to turn on system build-in AEC and it works as expected. The article you share are very interesting to read. Thanks

              – Akatsuki821
              Nov 28 '18 at 15:29
















            0














            I suggest to analyze the nature of the echo you are having in your system (for example test the echo path). Perhaps it is outside the capabilities of your built-in echo cancellation.






            share|improve this answer
























            • Thanks for giving suggestions. We figured out the way to turn on system build-in AEC and it works as expected. The article you share are very interesting to read. Thanks

              – Akatsuki821
              Nov 28 '18 at 15:29














            0












            0








            0







            I suggest to analyze the nature of the echo you are having in your system (for example test the echo path). Perhaps it is outside the capabilities of your built-in echo cancellation.






            share|improve this answer













            I suggest to analyze the nature of the echo you are having in your system (for example test the echo path). Perhaps it is outside the capabilities of your built-in echo cancellation.







            share|improve this answer












            share|improve this answer



            share|improve this answer










            answered Nov 22 '18 at 6:12









            TimTim

            342




            342













            • Thanks for giving suggestions. We figured out the way to turn on system build-in AEC and it works as expected. The article you share are very interesting to read. Thanks

              – Akatsuki821
              Nov 28 '18 at 15:29



















            • Thanks for giving suggestions. We figured out the way to turn on system build-in AEC and it works as expected. The article you share are very interesting to read. Thanks

              – Akatsuki821
              Nov 28 '18 at 15:29

















            Thanks for giving suggestions. We figured out the way to turn on system build-in AEC and it works as expected. The article you share are very interesting to read. Thanks

            – Akatsuki821
            Nov 28 '18 at 15:29





            Thanks for giving suggestions. We figured out the way to turn on system build-in AEC and it works as expected. The article you share are very interesting to read. Thanks

            – Akatsuki821
            Nov 28 '18 at 15:29













            0














            Finally we figured out how to turn on the iOS build-in AEC on Xamarin.iOS platform.
            The documentation here helped a lot to have a better understanding of the details of Audio Unit (even though this document is for iOS native development environment) and the working mechanism. If you want to precisely control every audio unit and understand parameters in the functions, this document is a must read.



            The work here provide a very nice start point to do experiment on Audio Unit. I made some modifications in step 2 of this post. The key part is that we have to use AudioTypeOutput.VoiceProcessingIO, this turns on AEC.



            public void prepareAudioUnit()
            {

            var _audioComponent = AudioComponent.FindComponent(AudioTypeOutput.VoiceProcessingIO);

            audioUnit = new AudioUnit.AudioUnit(_audioComponent);

            audioUnit.SetEnableIO(true,
            AudioUnitScopeType.Input,
            1 // Remote Input
            );

            // setting audio format
            audioUnit.SetAudioFormat(audioStreamBasicDesc,
            AudioUnitScopeType.Output,
            1
            );

            audioUnit.SetInputCallback(input_CallBack, AudioUnitScopeType.Input, 1);

            audioUnit.SetRenderCallback(render_CallBack, AudioUnitScopeType.Global, 0);


            audioUnit.Initialize();
            audioUnit.Start();
            }





            share|improve this answer




























              0














              Finally we figured out how to turn on the iOS build-in AEC on Xamarin.iOS platform.
              The documentation here helped a lot to have a better understanding of the details of Audio Unit (even though this document is for iOS native development environment) and the working mechanism. If you want to precisely control every audio unit and understand parameters in the functions, this document is a must read.



              The work here provide a very nice start point to do experiment on Audio Unit. I made some modifications in step 2 of this post. The key part is that we have to use AudioTypeOutput.VoiceProcessingIO, this turns on AEC.



              public void prepareAudioUnit()
              {

              var _audioComponent = AudioComponent.FindComponent(AudioTypeOutput.VoiceProcessingIO);

              audioUnit = new AudioUnit.AudioUnit(_audioComponent);

              audioUnit.SetEnableIO(true,
              AudioUnitScopeType.Input,
              1 // Remote Input
              );

              // setting audio format
              audioUnit.SetAudioFormat(audioStreamBasicDesc,
              AudioUnitScopeType.Output,
              1
              );

              audioUnit.SetInputCallback(input_CallBack, AudioUnitScopeType.Input, 1);

              audioUnit.SetRenderCallback(render_CallBack, AudioUnitScopeType.Global, 0);


              audioUnit.Initialize();
              audioUnit.Start();
              }





              share|improve this answer


























                0












                0








                0







                Finally we figured out how to turn on the iOS build-in AEC on Xamarin.iOS platform.
                The documentation here helped a lot to have a better understanding of the details of Audio Unit (even though this document is for iOS native development environment) and the working mechanism. If you want to precisely control every audio unit and understand parameters in the functions, this document is a must read.



                The work here provide a very nice start point to do experiment on Audio Unit. I made some modifications in step 2 of this post. The key part is that we have to use AudioTypeOutput.VoiceProcessingIO, this turns on AEC.



                public void prepareAudioUnit()
                {

                var _audioComponent = AudioComponent.FindComponent(AudioTypeOutput.VoiceProcessingIO);

                audioUnit = new AudioUnit.AudioUnit(_audioComponent);

                audioUnit.SetEnableIO(true,
                AudioUnitScopeType.Input,
                1 // Remote Input
                );

                // setting audio format
                audioUnit.SetAudioFormat(audioStreamBasicDesc,
                AudioUnitScopeType.Output,
                1
                );

                audioUnit.SetInputCallback(input_CallBack, AudioUnitScopeType.Input, 1);

                audioUnit.SetRenderCallback(render_CallBack, AudioUnitScopeType.Global, 0);


                audioUnit.Initialize();
                audioUnit.Start();
                }





                share|improve this answer













                Finally we figured out how to turn on the iOS build-in AEC on Xamarin.iOS platform.
                The documentation here helped a lot to have a better understanding of the details of Audio Unit (even though this document is for iOS native development environment) and the working mechanism. If you want to precisely control every audio unit and understand parameters in the functions, this document is a must read.



                The work here provide a very nice start point to do experiment on Audio Unit. I made some modifications in step 2 of this post. The key part is that we have to use AudioTypeOutput.VoiceProcessingIO, this turns on AEC.



                public void prepareAudioUnit()
                {

                var _audioComponent = AudioComponent.FindComponent(AudioTypeOutput.VoiceProcessingIO);

                audioUnit = new AudioUnit.AudioUnit(_audioComponent);

                audioUnit.SetEnableIO(true,
                AudioUnitScopeType.Input,
                1 // Remote Input
                );

                // setting audio format
                audioUnit.SetAudioFormat(audioStreamBasicDesc,
                AudioUnitScopeType.Output,
                1
                );

                audioUnit.SetInputCallback(input_CallBack, AudioUnitScopeType.Input, 1);

                audioUnit.SetRenderCallback(render_CallBack, AudioUnitScopeType.Global, 0);


                audioUnit.Initialize();
                audioUnit.Start();
                }






                share|improve this answer












                share|improve this answer



                share|improve this answer










                answered Nov 28 '18 at 15:40









                Akatsuki821Akatsuki821

                1114




                1114






























                    draft saved

                    draft discarded




















































                    Thanks for contributing an answer to Stack Overflow!


                    • Please be sure to answer the question. Provide details and share your research!

                    But avoid



                    • Asking for help, clarification, or responding to other answers.

                    • Making statements based on opinion; back them up with references or personal experience.


                    To learn more, see our tips on writing great answers.




                    draft saved


                    draft discarded














                    StackExchange.ready(
                    function () {
                    StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53402009%2facoustic-echo-cancellation-aec-on-xamarin-ios%23new-answer', 'question_page');
                    }
                    );

                    Post as a guest















                    Required, but never shown





















































                    Required, but never shown














                    Required, but never shown












                    Required, but never shown







                    Required, but never shown

































                    Required, but never shown














                    Required, but never shown












                    Required, but never shown







                    Required, but never shown







                    Popular posts from this blog

                    Guess what letter conforming each word

                    Port of Spain

                    Run scheduled task as local user group (not BUILTIN)