Track WebDriver requests with Fiddler or so
0
I am creating a web scraper application using C# and Selenium with Chrome WebDriver. So far I have configured it like this:
public class Program
{
private static ChromeOptions options;
private static ChromeDriver driver;
static void Main(string args)
{
var proxy = new Proxy();
proxy.Kind = ProxyKind.Manual;
proxy.IsAutoDetect = false;
proxy.HttpProxy = "127.0.0.1.fiddler:52840";
options = new ChromeOptions();
options.Proxy = proxy;
options.AddArgument("ignore-certificate-errors");
options.SetLoggingPreference(LogType.Driver, LogLevel.All);
driver = new ChromeDriver(options);
driver.Navigate().GoToUrl("https://www.google.com");
}
}
but I can't seem to get any info in Fiddler while accessing "https://www.google.com".
I am also wondering - if I install an application on my PC (I am using Windows 10 Pro) which enables VPN while browsing, will this VPN be active while scraping or I need a manual configuration in the WebDriver?
Thank you!
c# selenium selenium-chromedriver vpn options
asked Nov 16 '18 at 23:30
Radoslav StoyanovRadoslav Stoyanov
32
add a comment |
0
I am creating a web scraper application using C# and Selenium with Chrome WebDriver. So far I have configured it like this:
public class Program
{
private static ChromeOptions options;
private static ChromeDriver driver;
static void Main(string args)
{
var proxy = new Proxy();
proxy.Kind = ProxyKind.Manual;
proxy.IsAutoDetect = false;
proxy.HttpProxy = "127.0.0.1.fiddler:52840";
options = new ChromeOptions();
options.Proxy = proxy;
options.AddArgument("ignore-certificate-errors");
options.SetLoggingPreference(LogType.Driver, LogLevel.All);
driver = new ChromeDriver(options);
driver.Navigate().GoToUrl("https://www.google.com");
}
}
but I can't seem to get any info in Fiddler while accessing "https://www.google.com".
I am also wondering - if I install an application on my PC (I am using Windows 10 Pro) which enables VPN while browsing, will this VPN be active while scraping or I need a manual configuration in the WebDriver?
Thank you!
c# selenium selenium-chromedriver vpn options
asked Nov 16 '18 at 23:30
Radoslav StoyanovRadoslav Stoyanov
32
add a comment |
0
0
0
I am creating a web scraper application using C# and Selenium with Chrome WebDriver. So far I have configured it like this:
public class Program
{
private static ChromeOptions options;
private static ChromeDriver driver;
static void Main(string args)
{
var proxy = new Proxy();
proxy.Kind = ProxyKind.Manual;
proxy.IsAutoDetect = false;
proxy.HttpProxy = "127.0.0.1.fiddler:52840";
options = new ChromeOptions();
options.Proxy = proxy;
options.AddArgument("ignore-certificate-errors");
options.SetLoggingPreference(LogType.Driver, LogLevel.All);
driver = new ChromeDriver(options);
driver.Navigate().GoToUrl("https://www.google.com");
}
}
but I can't seem to get any info in Fiddler while accessing "https://www.google.com".
I am also wondering - if I install an application on my PC (I am using Windows 10 Pro) which enables VPN while browsing, will this VPN be active while scraping or I need a manual configuration in the WebDriver?
Thank you!
c# selenium selenium-chromedriver vpn options
asked Nov 16 '18 at 23:30
Radoslav StoyanovRadoslav Stoyanov
32
I am creating a web scraper application using C# and Selenium with Chrome WebDriver. So far I have configured it like this:
public class Program
{
private static ChromeOptions options;
private static ChromeDriver driver;
static void Main(string args)
{
var proxy = new Proxy();
proxy.Kind = ProxyKind.Manual;
proxy.IsAutoDetect = false;
proxy.HttpProxy = "127.0.0.1.fiddler:52840";
options = new ChromeOptions();
options.Proxy = proxy;
options.AddArgument("ignore-certificate-errors");
options.SetLoggingPreference(LogType.Driver, LogLevel.All);
driver = new ChromeDriver(options);
driver.Navigate().GoToUrl("https://www.google.com");
}
}
but I can't seem to get any info in Fiddler while accessing "https://www.google.com".
I am also wondering - if I install an application on my PC (I am using Windows 10 Pro) which enables VPN while browsing, will this VPN be active while scraping or I need a manual configuration in the WebDriver?
Thank you!
c# selenium selenium-chromedriver vpn options
c# selenium selenium-chromedriver vpn options
asked Nov 16 '18 at 23:30
Radoslav StoyanovRadoslav Stoyanov
32
asked Nov 16 '18 at 23:30
Radoslav StoyanovRadoslav Stoyanov
32
edited Nov 17 '18 at 9:30
Radoslav Stoyanov
asked Nov 16 '18 at 23:30
Radoslav StoyanovRadoslav Stoyanov
32
asked Nov 16 '18 at 23:30
Radoslav StoyanovRadoslav Stoyanov
32
asked Nov 16 '18 at 23:30
Radoslav StoyanovRadoslav Stoyanov
32
32
add a comment |
add a comment |
1 Answer
1
active
oldest
votes
0
By default fiddler will set Internet Options to use proxy and Chrome will follow it so there is no need to set it in your code. things to check:
- Check Fiddler listen port:
Tools > Options > Connections > Fiddler listens on ports
- make sure
127.0.0.1.fiddlervalid local domain or use local IP127.0.0.1
- Disable IPv6
answered Nov 17 '18 at 1:16
ewwinkewwink
11.5k22237
Hello @ewwink. Fiddler listens to port 8888. I have removed the proxy configuration part from my code and disabled IPv6. But I wasn't able to find out if127.0.0.1:fiddleris a valid local domain, because I don't know how. Now when I acces a website with Selenium, I can see the traffic in Fiddler. Thank you!
– Radoslav Stoyanov
Nov 17 '18 at 12:26
Hmmm, I am a bit confused - if Fiddler stays as a proxy, shouldn't it change my IP to the rest of the world? With or without this software I end up using one and the same IP.
– Radoslav Stoyanov
Nov 17 '18 at 19:28
your public IP will not change and if fiddler closed it will restored to default internet option.
– ewwink
Nov 17 '18 at 19:35
add a comment |
Your Answer
StackExchange.ifUsing("editor", function () {
StackExchange.using("externalEditor", function () {
StackExchange.using("snippets", function () {
StackExchange.snippets.init();
});
});
}, "code-snippets");
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "1"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
draft saved
draft discarded
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53346701%2ftrack-webdriver-requests-with-fiddler-or-so%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
0
By default fiddler will set Internet Options to use proxy and Chrome will follow it so there is no need to set it in your code. things to check:
- Check Fiddler listen port:
Tools > Options > Connections > Fiddler listens on ports
- make sure
127.0.0.1.fiddlervalid local domain or use local IP127.0.0.1
- Disable IPv6
answered Nov 17 '18 at 1:16
ewwinkewwink
11.5k22237
Hello @ewwink. Fiddler listens to port 8888. I have removed the proxy configuration part from my code and disabled IPv6. But I wasn't able to find out if127.0.0.1:fiddleris a valid local domain, because I don't know how. Now when I acces a website with Selenium, I can see the traffic in Fiddler. Thank you!
– Radoslav Stoyanov
Nov 17 '18 at 12:26
Hmmm, I am a bit confused - if Fiddler stays as a proxy, shouldn't it change my IP to the rest of the world? With or without this software I end up using one and the same IP.
– Radoslav Stoyanov
Nov 17 '18 at 19:28
your public IP will not change and if fiddler closed it will restored to default internet option.
– ewwink
Nov 17 '18 at 19:35
add a comment |
0
By default fiddler will set Internet Options to use proxy and Chrome will follow it so there is no need to set it in your code. things to check:
- Check Fiddler listen port:
Tools > Options > Connections > Fiddler listens on ports
- make sure
127.0.0.1.fiddlervalid local domain or use local IP127.0.0.1
- Disable IPv6
answered Nov 17 '18 at 1:16
ewwinkewwink
11.5k22237
Hello @ewwink. Fiddler listens to port 8888. I have removed the proxy configuration part from my code and disabled IPv6. But I wasn't able to find out if127.0.0.1:fiddleris a valid local domain, because I don't know how. Now when I acces a website with Selenium, I can see the traffic in Fiddler. Thank you!
– Radoslav Stoyanov
Nov 17 '18 at 12:26
Hmmm, I am a bit confused - if Fiddler stays as a proxy, shouldn't it change my IP to the rest of the world? With or without this software I end up using one and the same IP.
– Radoslav Stoyanov
Nov 17 '18 at 19:28
your public IP will not change and if fiddler closed it will restored to default internet option.
– ewwink
Nov 17 '18 at 19:35
add a comment |
0
0
0
By default fiddler will set Internet Options to use proxy and Chrome will follow it so there is no need to set it in your code. things to check:
- Check Fiddler listen port:
Tools > Options > Connections > Fiddler listens on ports
- make sure
127.0.0.1.fiddlervalid local domain or use local IP127.0.0.1
- Disable IPv6
answered Nov 17 '18 at 1:16
ewwinkewwink
11.5k22237
By default fiddler will set Internet Options to use proxy and Chrome will follow it so there is no need to set it in your code. things to check:
- Check Fiddler listen port:
Tools > Options > Connections > Fiddler listens on ports
- make sure
127.0.0.1.fiddlervalid local domain or use local IP127.0.0.1
- Disable IPv6
answered Nov 17 '18 at 1:16
ewwinkewwink
11.5k22237
answered Nov 17 '18 at 1:16
ewwinkewwink
11.5k22237
answered Nov 17 '18 at 1:16
ewwinkewwink
11.5k22237
answered Nov 17 '18 at 1:16
ewwinkewwink
11.5k22237
11.5k22237
Hello @ewwink. Fiddler listens to port 8888. I have removed the proxy configuration part from my code and disabled IPv6. But I wasn't able to find out if127.0.0.1:fiddleris a valid local domain, because I don't know how. Now when I acces a website with Selenium, I can see the traffic in Fiddler. Thank you!
– Radoslav Stoyanov
Nov 17 '18 at 12:26
Hmmm, I am a bit confused - if Fiddler stays as a proxy, shouldn't it change my IP to the rest of the world? With or without this software I end up using one and the same IP.
– Radoslav Stoyanov
Nov 17 '18 at 19:28
your public IP will not change and if fiddler closed it will restored to default internet option.
– ewwink
Nov 17 '18 at 19:35
add a comment |
Hello @ewwink. Fiddler listens to port 8888. I have removed the proxy configuration part from my code and disabled IPv6. But I wasn't able to find out if127.0.0.1:fiddleris a valid local domain, because I don't know how. Now when I acces a website with Selenium, I can see the traffic in Fiddler. Thank you!
– Radoslav Stoyanov
Nov 17 '18 at 12:26
Hmmm, I am a bit confused - if Fiddler stays as a proxy, shouldn't it change my IP to the rest of the world? With or without this software I end up using one and the same IP.
– Radoslav Stoyanov
Nov 17 '18 at 19:28
your public IP will not change and if fiddler closed it will restored to default internet option.
– ewwink
Nov 17 '18 at 19:35
Hello @ewwink. Fiddler listens to port 8888. I have removed the proxy configuration part from my code and disabled IPv6. But I wasn't able to find out if
127.0.0.1:fiddler is a valid local domain, because I don't know how. Now when I acces a website with Selenium, I can see the traffic in Fiddler. Thank you!– Radoslav Stoyanov
Nov 17 '18 at 12:26
Hello @ewwink. Fiddler listens to port 8888. I have removed the proxy configuration part from my code and disabled IPv6. But I wasn't able to find out if
127.0.0.1:fiddler is a valid local domain, because I don't know how. Now when I acces a website with Selenium, I can see the traffic in Fiddler. Thank you!– Radoslav Stoyanov
Nov 17 '18 at 12:26
Hmmm, I am a bit confused - if Fiddler stays as a proxy, shouldn't it change my IP to the rest of the world? With or without this software I end up using one and the same IP.
– Radoslav Stoyanov
Nov 17 '18 at 19:28
Hmmm, I am a bit confused - if Fiddler stays as a proxy, shouldn't it change my IP to the rest of the world? With or without this software I end up using one and the same IP.
– Radoslav Stoyanov
Nov 17 '18 at 19:28
your public IP will not change and if fiddler closed it will restored to default internet option.
– ewwink
Nov 17 '18 at 19:35
your public IP will not change and if fiddler closed it will restored to default internet option.
– ewwink
Nov 17 '18 at 19:35
add a comment |
draft saved
draft discarded
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
draft saved
draft discarded
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53346701%2ftrack-webdriver-requests-with-fiddler-or-so%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
