4

I tried doing something like this but it doesn't work!

$ch = curl_init(); 
curl_setopt($ch, CURLOPT_URL, "http://google.com/"); 
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1); 
curl_setopt($ch, CURLOPT_HTTP_VERSION, CURL_HTTP_VERSION_1_0);
curl_setopt($ch, CURLOPT_HTTPHEADER, array('GET /search?q=kk HTTP/1.1
Host: www.google.de
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:6.0.2) Gecko/20100101 Firefox/6.0.2
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-gb,en;q=0.5
Accept-Encoding: gzip, deflate
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
Proxy-Connection: Close
Cookie: PREF=ID=2bb051bfbf00e95b:U=c0bb6046a0ce0334:
Cache-Control: max-age=0
Connection: Close
'));
$response = curl_exec($ch); 
curl_close($ch);   



    echo $response;

Also, Is it possible to make the entire request with just headers without setting the URL? I mean without this?

curl_setopt($ch, CURLOPT_URL, "http://google.com/"); 

Thanks!

4
  • Are you trying to set an array in the setopt() implicitly? or something different? Commented Mar 2, 2012 at 19:51
  • Why are you not separating the header options with a comma like explained in the manual? Commented Mar 2, 2012 at 19:53
  • @Xepoch I'm trying to use the whole stream as headers. . . Commented Mar 2, 2012 at 20:04
  • From the manual: An array of HTTP header fields to set, in the format array('Content-type: text/plain', 'Content-length: 100'). They are separated by comma while yours is one big entry. Theres also some comments in the manual that show the same. Commented Mar 2, 2012 at 20:07

5 Answers 5

5

I got it to work.

1) Change header Host: www.google.de to Host: www.google.com

Motivation: the host specified in the Host header should match the URL's host exactly.

2) Use "www.google.com" instead of "google.com"

Motivation: search requests made to google.com will not retrieve search results. You will be told to go to www.google.com.

3) Set the full URL into CURLOPT_URL, not just the hostname. E.g. change the CURLOPT_URL to curl_setopt($ch, CURLOPT_URL, "http://www.google.com/search?q=kk");

Motivation: correct usage of cURL API.

4) Remove GET /search?q=kk HTTP/1.1 from CURLOPT_HTTPHEADER -- it's misplaced.

Motivation: correct usage of cURL API.

5) The response will be gzip or deflate compressed. To stop this, remove the Accept-Encoding: gzip, deflate request header.

Motivation: if you tell Google you're capable of receiving a compressed response, they will send you one. Decompressing an HTTP response is an extra step that you may not want to undertake. It may be easier to deal with the response if it is in an uncompressed text form.

Sign up to request clarification or add additional context in comments.

Comments

4

To add onto what the other posters have said, you also cannot stick a GET command in the CURLOPT_HTTPHEADER array, because that's specified in the other cURL options. cURL is meant to be operated using the curl_setopt function; you cannot bypass it by putting your HTTP message in the headers section. For instance, to ensure that your command is an HTTP GET operation, you set CURLOPT_HTTPGET to TRUE (although by default cURL will send a GET until you change it to something else).

To address your question of why you can't get to the right URL, that's because you need to specify the entire pathname in CURLOPT_URL, not just the host. So, you should really be writing curl_setopt($ch, CURLOPT_URL, "http://google.de/search?q=kk HTTP/1.1"); to set the URL.

Further, I have no idea why you put Connection: Close in he HTTP headers for a GET request. In that header, you're telling Google that you're closing the connection you have.; this is handled by curl_close($ch);, so forget about that header. In fact, half the items in your HTTP headers have no place being there. For instance, why are you sending a cookie in a request to get search results? Make sure you know what each header does before you send it out. Otherwise, you have absolutely no way to tell if you are sending the correct headers or not.

Comments

2

You've got a few problems, but they should be easy to sort out. First, you're setting the host in the header different than the host in the URL request, but since you're doing HTTP1.0, that isn't needed anyways.

Second, you need each line in the HTTPHEADER as its own thing in the array, and you don't include the GET line.

curl_setopt($ch, CURLOPT_HTTPHEADER, array(
   'User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:6.0.2) Gecko/20100101 Firefox/6.0.2',
    'Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8',
    'Accept-Language: en-gb,en;q=0.5',
    'Accept-Encoding: gzip, deflate',
    'Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7',
    'Proxy-Connection: Close',
    'Cookie: PREF=ID=2bb051bfbf00e95b:U=c0bb6046a0ce0334:',
    'Cache-Control: max-age=0',
    'Connection: Close'
));

(You clearly stole this from Firefox, and an old version at that, but we'll let it slide.) Finally, yes, you must specify CURLOPT_URL, that's just the way the cURL API is designed.

Comments

1

If you need such a high level of control over the resulting HTTP request, I would recommend using raw socket functions to manually send the request instead. The manual even has an example of doing a HTTP request with fsockets:

$fp = fsockopen("www.example.com", 80, $errno, $errstr, 30);
if (!$fp) {
    echo "$errstr ($errno)<br />\n";
} else {
    $out = "GET / HTTP/1.1\r\n";
    $out .= "Host: www.example.com\r\n";
    $out .= "Connection: Close\r\n\r\n";
    fwrite($fp, $out);
    while (!feof($fp)) {
        echo fgets($fp, 128);
    }
    fclose($fp);
}

1 Comment

Fsockopen is disabled. That's why I am using cURL!
1

You should read the fine manual more carefully next time. There is an example how to add the fields: array('Content-type: text/plain', 'Content-length: 100'), not everything in a one string, but fields are separate array elements.

Comments

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.