How would I use JavaScript to retrieve text from a different website?

Question

Disclaimer: please excuse the religious aspects of this post.

Anyway, the title says what I'm looking for essentially. I'm writing a program to find Bible references on a page, then display the actual verse in a tooltip upon mouseover of the reference.

I'm hoping to get permission from this site eventually. Now, I've done research and I learned about the "Same Origin Policy"; but as far as I can tell this doesn't really apply because I'm not trying to call functions/methods from JavaScript on that site.

All I want to do is grab a verse off of the page. I thought this site would be convenient because it will mark individual verses with the correct url like so: http://www.drbo.org/x/d?b=drb&bk=47&ch=16&l=18. I've tried loading this up in an iframe but I couldn't for the life of me get hold of the the <span> element with class='highlight' which is, by chance, the only span on the page.

The attempt I just made today was to use JSONP in the form of:

function crossedFingers(variable) {
    alert(variable);
}

var scr = document.createElement('script');
scr.setAttribute('type', 'text/javascript');
scr.setAttribute('charset', 'utf-8');
scr.setAttribute('src', 'http://www.drbo.org/x/d?b=drb&bk=47&ch=16&l=18&callback=crossedFingers');

document.getElementsByTagName('head').item(0).appendChild(scr);

I also had a version where I had

function crossedFingers() {
    alert('hi');
}

just to see if the function actually got called and it didn't. I'm using WAMP and I have these files in the www directory, but I'm not sure that should effect it. Any help would be appreciated, sorry if this in an inappropriate question!

I've just been frustrated. @_@

NullUserException · Accepted Answer · 2011-10-20 05:16:30Z

4

This is exactly what the Same Origin Policy tries to prevent. It won't let you do it, pure and simple. Imagine if you could "just retrieve text" from an online banking website... That wouldn't go well.

JSONP only works when the other website allows it. Even then, you are leaving yourself open to a host of vulnerabilities. For starters, this allows the target site to inject arbitrary code into your website. You should only use this if you absolutely trust the other website.

A better solution is to use regular AJAX and have your server fetch the contents from the remote website (eg: using cURL) and relay them back to the client.

answered Oct 20, 2011 at 5:16

NullUserException

85.8k31 gold badges212 silver badges239 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

dunnza Over a year ago

Thank you, your example makes things much more clear to me! So my goal if for someone to toss something like <script src='www.example.com/path/to/app.js'></script> on their web pages to get it to work. If, hypothetically, I could get the owner of that website to host the file on their server, would I be able to access the text no problem, then?

James Williams · Accepted Answer · 2011-10-20 05:16:27Z

Javascript is a client-side language and normally cannot make cross-domain queries. With AJAX it was possible to do it with something like the following code.

<iframe id="holder" style="display: none">

<script type="text/javascript">
$("#holder").load("http://www.drbo.org/x/d?b=drb&bk=47&ch=16&l=18",function () {
 $link = $("#holder").contents().find("span.highlight");
});
</script>

And then using javascript to disect the elements. But with recent cross-domain policies this has been getting harder. You may have to change and use a server-side language like php to do a curl request and grab the information. PHP would be quicker.

Miquel Capó Casasnovas · Accepted Answer · 2023-09-20 16:00:24Z

Here a simple async function to retrieve the document

function urlToHTML(url){

    return new Promise((res, rej)=>{

        let iframe = document.createElement("iframe")

        iframe.style.visibility = "hidden"
       
        document.body.appendChild(iframe)
    
        iframe.addEventListener("load", ()=>{
            res(iframe.ownerDocument)
        })

       iframe.src = url
    

    })
   
    

}


urlToHTML(url)
.then(doc=>{
// do something with the document
})

Collectives™ on Stack Overflow

How would I use JavaScript to retrieve text from a different website?

3 Answers 3

1 Comment

Comments

Comments

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

1 Comment

Comments

Comments

Related