html parsing with php simple_html_dom

Question

I'm parsing internet newspapers's columinst page. I have problem about this site

http://www.sozcu.com.tr/kategori/yazarlar/

the parsing was working fine in the starting but it stopped working.

Here's my code

$curl_handle=curl_init();
curl_setopt($curl_handle, CURLOPT_URL,$gazeteAdress);
//curl_setopt($curl_handle, CURLOPT_CONNECTTIMEOUT, 2);
curl_setopt($curl_handle, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($curl_handle, CURLOPT_USERAGENT, 'mozilla');
$query = curl_exec($curl_handle);
curl_close($curl_handle);
$html = new simple_html_dom();
$html->load($query);

I don't know why my code sometimes is not parsing the site, so I was thinking about connection_timeout. But It is not the problem, so I was thinking of printing html page with curl instead.

echo $html;

Here is result. (sometimes my code is not parsing html page properly)

why the html tags are not coming and why am seeing the result like this. Can anyone help ?

Fedir Petryk · Accepted Answer · 2015-10-12 10:24:48Z

1

The content is returned compressed so you should specify Accept-Encoding with 'gzip,deflate' header for curl.

Please add this line
curl_setopt($curl_handle, CURLOPT_ENCODING, "gzip,deflate");
after this
curl_setopt($curl_handle, CURLOPT_USERAGENT, 'mozilla');

answered Oct 12, 2015 at 10:24

Fedir Petryk

4953 silver badges17 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

user1919915 · Accepted Answer · 2015-10-12 09:39:10Z

0

Add this on top of your php script

header('Content-Type: text/html; charset=utf-8');

answered Oct 12, 2015 at 9:39

user1919915

131 silver badge7 bronze badges

3 Comments

Ucdemir Over a year ago

have this: <meta http-equiv="Content-Type" content="text/HTML; charset=utf-8" />

user1919915 Over a year ago

You don't need this, only <?php header('Content-Type: text/html; charset=utf-8');

Ucdemir Over a year ago

mate sometimes it is work but sometimes it is not.. It is not the Turkish character problem

Collectives™ on Stack Overflow

html parsing with php simple_html_dom

2 Answers 2

Comments

3 Comments

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

3 Comments

Related