How to access the devtools iframe-selection menu with Selenium Webdriver?

Developer Tools
#1

I am trying to do what the title says, using the Selenium Webdriver Python package.

If I open a page that embeds an iframe in CONTEXT_CHROME I can see the element that I think corresponds to the iframe-selection menu:

<browser type="content" flex="1" class="devtools-toolbox-bottom-iframe" height="250" aria-label="Developer Tools" src="about:devtools-toolbox" session_id="910"/>

My problem is that I do not know what to do with it afterwards to actually access the iframe menu: it doesn’t respond to clicks, and it is not itself an iframe, so I cannot “switch” to it.

PS

There is a Stack Overflow post.

#2

@achirvasub

Hi! Looking at the SO post you linked, I ended up on https://stackoverflow.com/questions/63680269/is-it-possible-to-emulate-the-firefox-console-cd-function-and-or-javascript-cont/63715117#63715117

Is this why you are trying to use devtools here? In order to run JavaScript code in the context of a specific frame?

If that’s the case did you try to use switchTo + executeScript from webdriver, without using devtools? @whimboo mentioned that this should allow you to run script in the frame.

#3

I am trying to achieve exactly the same thing I was in that even older post, except that nothing’s working anymore: I can no longer send keys to any elements in CONTEXT_CHROME (I get errors to the effect that they are not reachable by keyboard), the console cd function has been deprecated, plus the issue I initially reported.

As to your question: I’m afraid I do not understand. There is no script. I simply want access to the frame-selectiom menu. If it’s doable in a browser with a mouse, surely it should be doable with Webdriver (which in turn used to be the case…).

#4

The question is why do you use DevTools in this test?

It seems like you had to use this as a workaround but you should not have to go through DevTools for your test (and that’s not something we try to maintain so it’s bound to fail regularly whenever we change something).

Reading the old post again: " I have a page loading a cross-origin iframe I have no control over, and would like to access DOM elements under that iframe.".
It seems to me that webdriver should be able to interact with cross origin iframes, and that’s what should be fixed here.

1 Like
#5

Ah: as I recall, back then these two types of action would show me different DOMs, for reasons that I never elucidated:

  • staying in CONTEXT_CONTENT (the default) and switching to that iframe
  • switching to CONTEXT_CHROME and going through all of that devtools rigamarole.

I was not getting back the same elements when printing out the full contents of what WebDriver was seeing (with print(webdriver.page_source)): the former was not giving me access to the buttons I needed to click, etc.

I can try that again and report back: I have to confess I did not attempt it this time around…

#6

Reporting back: this worked, exactly as suggested by @jdescottes (thank you!):

  • staying in the usual context (webdriver.CONTEXT_CONTENT)
  • I can switch to my target iframe:
ifr = webdriver.find_element(By.ID, <whatever>)
webdriver.switch_to.frame(ifr)
  • and then control the browser with javascript:
js_command = f'btn=document.getElementById(<login-button-id>); btn.click(); fld = document.getElementsByClassName("passcode-input")[0]; fld.value = <passcode>; btn.click(); '
webdriver.execute_script(js_command)

An aside on an ensuing point of confusion: finding that button with javascript and clicking it works fine, as mentioned, but trying to click directly with webdriver does nothing:

btn = webdriver.find_element(By.ID,<login-button-id>)
btn.click()

does nothing. There’s no error either, meaning that the button is located; the problem is the click isn’t working. Any idea why there would be this difference in behavior between the js and webdriver clicks?

#7

Hi @achirvasub. So it looks like the information that Julian has given already helped you to find the button element. Nevertheless it seems that you cannot click it. To figure out what’s happening here we would need as best two things from you:

  1. If you could create a minimized HTML testcase that reproduces the problem, or give us the actual URLs that you were working with would be good.

  2. You want to create a trace log when running geckodriver, and attach it here. It could already give us an indication what might be wrong.

Possible issues here might be a lazy setup of the button, a custom click handler, that it might be located behind some other element that we do not correctly detect.

Thanks in advance.

Henrik

#8

Thank you very much for following up @whimboo. I unfortunately cannot provide as much detail as I would like, since

  • I only have this one instance of the problem
  • which arises while trying to automate 2-factor authentication through Duo Security (nothing nefarious; my company is mandating this, and I am trying to automate these hoops I have to jump through…).

I can show a fragment of the DOM I see in the Duo iframe, in a pastebin (not all of it; parts of the DOM display my home IP, other parts reference what seem to be semi-permanent ID numbers used internally by Duo, etc., so some lines are marked REDACTED).

The button in question is “the one” with id="passcode", but you’ll see it appears twice, hence the scare quotes here. I am not well-versed enough in the subtleties of how a DOM might be put together, but I thought the id was supposed to be unique.

In any event, the behavior I am seeing is as follows (with the code referencing a perl implementation of the driver bindings; this is what I happen to be using at the moment):

  • if I run js all is well, and I can click the button (or presumably its first instance): I can simply pass the string
btn=document.getElementById("passcode"); 
btn.click();

for webdriver to run as a script for me, with $driver->script(<that-whole-thing-as-a-string>);

  • if I search for all elements with id="passcode" directly with webdriver (in its perl incarnation this means my @btns=$driver->find_id("passcode")), then I can

    • click the first element found in that array with $btns[0]->click() to absolutely no effect,
    • or click the second one with $btns[1]->click() to receive a complaint:
    element not interactable: Element <button id="passcode" class="positive auth-button" type="submit"> could not be scrolled into view
#9

Incidentally, I realized just now that in fact switching to the iframe will not do after all…

That iframe login session sets a cookie from the domain embedding that iframe (not the one I started out with). I cannot get access to that cookie from within the iframe (webdriver still sees me as being in the original domain, and returns only its cookies; I also cannot set cookies from the iframe-embedding domain I have saved from previous logins).

So no, this does not seem to do what I want…

Edit:

RE: accessing cookies, I have found an old bug report that seems to still be open. Am I correct that this still an issue @whimboo?