iOS - fully automate testing process with backups - is it possible?





.everyoneloves__top-leaderboard:empty,.everyoneloves__mid-leaderboard:empty,.everyoneloves__bot-mid-leaderboard:empty{ height:90px;width:728px;box-sizing:border-box;
}







0















The Story:

We have hundreds of devices running UI and unit tests. The tests are written using XCTest framework, and run on our applications, as well as system (Safari/Settings) and third party ones.



As part of the testing framework, we need to bring devices to a certain state before running each test. This includes clearing caches, changing setup items, installing applications etc. We currently use a combination of Apple Configurator, MDM, configuration profiles and UI tests to get to the desired state for each test.



The Problem:

So far so good, but the process of setting everything up is really costly, unstable and hard to maintain across different iOS versions.



The Idea:

Then we thought - why not use back ups? We can save a backup for each desired initial state, restore to the desired backup and start the test right away.



BUT...

There is one tiny problem with that idea - restoring from backup brings the device to the "Setup" state, where manual user intervention is required. Even if we use Apple Configurator to gain the device's trust, set up WiFi, and skip all possible setup items, the user must still click home button once in the "Hello" page, and then click "Get Started" on screen in order to get to SpringBoard and be able to start XCTest / application launching or debugging.



The Question:

Is obvious now: can we somehow automate the entire flow described above? That is - restore an iOS device from backup, skip the setup, and run XCTests.










share|improve this question





























    0















    The Story:

    We have hundreds of devices running UI and unit tests. The tests are written using XCTest framework, and run on our applications, as well as system (Safari/Settings) and third party ones.



    As part of the testing framework, we need to bring devices to a certain state before running each test. This includes clearing caches, changing setup items, installing applications etc. We currently use a combination of Apple Configurator, MDM, configuration profiles and UI tests to get to the desired state for each test.



    The Problem:

    So far so good, but the process of setting everything up is really costly, unstable and hard to maintain across different iOS versions.



    The Idea:

    Then we thought - why not use back ups? We can save a backup for each desired initial state, restore to the desired backup and start the test right away.



    BUT...

    There is one tiny problem with that idea - restoring from backup brings the device to the "Setup" state, where manual user intervention is required. Even if we use Apple Configurator to gain the device's trust, set up WiFi, and skip all possible setup items, the user must still click home button once in the "Hello" page, and then click "Get Started" on screen in order to get to SpringBoard and be able to start XCTest / application launching or debugging.



    The Question:

    Is obvious now: can we somehow automate the entire flow described above? That is - restore an iOS device from backup, skip the setup, and run XCTests.










    share|improve this question

























      0












      0








      0








      The Story:

      We have hundreds of devices running UI and unit tests. The tests are written using XCTest framework, and run on our applications, as well as system (Safari/Settings) and third party ones.



      As part of the testing framework, we need to bring devices to a certain state before running each test. This includes clearing caches, changing setup items, installing applications etc. We currently use a combination of Apple Configurator, MDM, configuration profiles and UI tests to get to the desired state for each test.



      The Problem:

      So far so good, but the process of setting everything up is really costly, unstable and hard to maintain across different iOS versions.



      The Idea:

      Then we thought - why not use back ups? We can save a backup for each desired initial state, restore to the desired backup and start the test right away.



      BUT...

      There is one tiny problem with that idea - restoring from backup brings the device to the "Setup" state, where manual user intervention is required. Even if we use Apple Configurator to gain the device's trust, set up WiFi, and skip all possible setup items, the user must still click home button once in the "Hello" page, and then click "Get Started" on screen in order to get to SpringBoard and be able to start XCTest / application launching or debugging.



      The Question:

      Is obvious now: can we somehow automate the entire flow described above? That is - restore an iOS device from backup, skip the setup, and run XCTests.










      share|improve this question














      The Story:

      We have hundreds of devices running UI and unit tests. The tests are written using XCTest framework, and run on our applications, as well as system (Safari/Settings) and third party ones.



      As part of the testing framework, we need to bring devices to a certain state before running each test. This includes clearing caches, changing setup items, installing applications etc. We currently use a combination of Apple Configurator, MDM, configuration profiles and UI tests to get to the desired state for each test.



      The Problem:

      So far so good, but the process of setting everything up is really costly, unstable and hard to maintain across different iOS versions.



      The Idea:

      Then we thought - why not use back ups? We can save a backup for each desired initial state, restore to the desired backup and start the test right away.



      BUT...

      There is one tiny problem with that idea - restoring from backup brings the device to the "Setup" state, where manual user intervention is required. Even if we use Apple Configurator to gain the device's trust, set up WiFi, and skip all possible setup items, the user must still click home button once in the "Hello" page, and then click "Get Started" on screen in order to get to SpringBoard and be able to start XCTest / application launching or debugging.



      The Question:

      Is obvious now: can we somehow automate the entire flow described above? That is - restore an iOS device from backup, skip the setup, and run XCTests.







      ios automation backup xctest apple-configurator






      share|improve this question













      share|improve this question











      share|improve this question




      share|improve this question










      asked Nov 21 '18 at 21:03









      ElistElist

      3,27422358




      3,27422358
























          0






          active

          oldest

          votes












          Your Answer






          StackExchange.ifUsing("editor", function () {
          StackExchange.using("externalEditor", function () {
          StackExchange.using("snippets", function () {
          StackExchange.snippets.init();
          });
          });
          }, "code-snippets");

          StackExchange.ready(function() {
          var channelOptions = {
          tags: "".split(" "),
          id: "1"
          };
          initTagRenderer("".split(" "), "".split(" "), channelOptions);

          StackExchange.using("externalEditor", function() {
          // Have to fire editor after snippets, if snippets enabled
          if (StackExchange.settings.snippets.snippetsEnabled) {
          StackExchange.using("snippets", function() {
          createEditor();
          });
          }
          else {
          createEditor();
          }
          });

          function createEditor() {
          StackExchange.prepareEditor({
          heartbeatType: 'answer',
          autoActivateHeartbeat: false,
          convertImagesToLinks: true,
          noModals: true,
          showLowRepImageUploadWarning: true,
          reputationToPostImages: 10,
          bindNavPrevention: true,
          postfix: "",
          imageUploader: {
          brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
          contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
          allowUrls: true
          },
          onDemand: true,
          discardSelector: ".discard-answer"
          ,immediatelyShowMarkdownHelp:true
          });


          }
          });














          draft saved

          draft discarded


















          StackExchange.ready(
          function () {
          StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53420448%2fios-fully-automate-testing-process-with-backups-is-it-possible%23new-answer', 'question_page');
          }
          );

          Post as a guest















          Required, but never shown

























          0






          active

          oldest

          votes








          0






          active

          oldest

          votes









          active

          oldest

          votes






          active

          oldest

          votes
















          draft saved

          draft discarded




















































          Thanks for contributing an answer to Stack Overflow!


          • Please be sure to answer the question. Provide details and share your research!

          But avoid



          • Asking for help, clarification, or responding to other answers.

          • Making statements based on opinion; back them up with references or personal experience.


          To learn more, see our tips on writing great answers.




          draft saved


          draft discarded














          StackExchange.ready(
          function () {
          StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53420448%2fios-fully-automate-testing-process-with-backups-is-it-possible%23new-answer', 'question_page');
          }
          );

          Post as a guest















          Required, but never shown





















































          Required, but never shown














          Required, but never shown












          Required, but never shown







          Required, but never shown

































          Required, but never shown














          Required, but never shown












          Required, but never shown







          Required, but never shown







          Popular posts from this blog

          Guess what letter conforming each word

          Port of Spain

          Run scheduled task as local user group (not BUILTIN)