Here's a tip you can try next time you have a problem like this. Maybe your task was somehow resistant to this, in which case I really do understand your pain and am not trying to paper it over, but just in case this could benefit you or folks like you:
In Chrome:
1) Right click on element you want to grab all of the contents of, and select Inspect Element. NOTE: If this element is the sort that isn't properly placed in the DOM and/or disappears when focus/mouse leaves, you can freeze the current DOM...but the right way to do that will depend on your context.
2) In Elements tab in Dev tools, the element should now be highlighted. You may need a parent or child element. But hopefully you can find it nearby and verify you have the right one with the Inspect tooling.
3) Right click on the element that has all your data in the DOM. Select Copy. Sometimes it will be enough to just Copy Element and paste it somewhere where you can start organizing the data. But for your task, it sounds like you might want to write a script for this so you can do it across multiple Elements. So, try Copy > Copy JS Path.
4) You now have access to the DOM element, even if it was created dynamically by JS and exists in the Shadow DOM. Try "paste" in the Console tab of Chrome DevTools.
5) The thing you grabbed might not be exactly the right one. Navigate down to the level you want to iterate across with the "children" attribute. Example:
> 1) Right click on element you want to grab all of the contents of, and select Inspect Element. NOTE: If this element is the sort that isn't properly placed in the DOM and/or disappears when focus/mouse leaves, you can freeze the current DOM...but the right way to do that will depend on your context.
Just to extend a touch—I use CMD+Shift+C or CTRL+Shift+C countless times in a day while debugging, or sometimes for this very reason. The shortcut is the same across browsers—at least Chrome/Edge/FF/Safari.
Instead of right clicking, it'll give you a "target" for selecting an element and highlight it before clicking.
In the console the selected element will be immediately accessible by `$0`.
In Chrome:
1) Right click on element you want to grab all of the contents of, and select Inspect Element. NOTE: If this element is the sort that isn't properly placed in the DOM and/or disappears when focus/mouse leaves, you can freeze the current DOM...but the right way to do that will depend on your context.
2) In Elements tab in Dev tools, the element should now be highlighted. You may need a parent or child element. But hopefully you can find it nearby and verify you have the right one with the Inspect tooling.
3) Right click on the element that has all your data in the DOM. Select Copy. Sometimes it will be enough to just Copy Element and paste it somewhere where you can start organizing the data. But for your task, it sounds like you might want to write a script for this so you can do it across multiple Elements. So, try Copy > Copy JS Path.
4) You now have access to the DOM element, even if it was created dynamically by JS and exists in the Shadow DOM. Try "paste" in the Console tab of Chrome DevTools.
5) The thing you grabbed might not be exactly the right one. Navigate down to the level you want to iterate across with the "children" attribute. Example:
6) You now have an HTMLCollection. You could be done! See if this gives you what you need: -7) Unfortunately, HTMLCollections are only Array-like. Fortunately, it's very easy to convert. -8) Now that you have a proper Array filled with HTML elements, you can do any kind of processing you need to grab the data you want. Hope this helps somebody write a script to remove the tedium from a data extraction job.