CookieJar in CURL – It Sucks

I was working with Linked in authentication management these days for one of my project where I have to loginto linked in using user’s credentials and fetch personal information and then display it in different form. My code was working properly in local machine, 3 different LAMP servers and one windows server. But finally when I deployed the code in production box, it fails. I quickly found that the cookiejar was not created for some permission problem. I tried to figure out what went wrong but I cant.

1. My script has the permission to create file in that directory on the fly

so there shouldn’t be any problem with this cookie jar and curl, but there was. So hours after hours i spend on it to find the reason and finally I decide to go without cookie jar. And If I don’t use cookie Jar, I have to manually parse the cookies sent to me after login and then set those cookies to my next request so that Linked in recognize me as a “coming back call after login”. I did that and my script worked pretty fine.

Fuck cookie jar in curl. Why the hell the developers didn’t provide a way to override to manage cookies??

If you are interested to know how did I solve it, let me explain a bit.

I set CURLSETOPT_HEADER to true so that I get back the header info
curl_setopt($ch, CURLOPT_HEADER, 1);

and then I parse that header info and extracted the cookies
$end = strpos($header, “Content-Type”);
$start = strpos($header, “Set-Cookie”);
$parts = split(“Set-Cookie: “,substr($header, $start, $end-$start));
$cookies = array();
foreach ($parts as $co)
{
$cd = split(“;”,$co);
if (!empty($cd[0]))
$cookies[] = $cd[0];
}

I will replace this section with RegEx

and finally I set those cookies to my next request using CURLSETOPT_COOKIE
curl_setopt($ch, CURLOPT_COOKIE, implode(“;”,$cookies));

Thats it!! It works pretty fine without cookie jar.

NOTE: I know that cookiejar is a very useful feature for curl users as it automates the cookie management. But I am saying “fuck cookiejar” because developers of curl didn’t provide any way to override cookie management process. If they give us way to use cookijar with any other options beside disk files, it would be beautiful. But in fact I am a big fan of this cookijar feature of curl, except the selfish automation.

About these ads

23 thoughts on “CookieJar in CURL – It Sucks

  1. nice say, libcurl was intended for command based processing. though libcurl has a feature to set your custom cookie handler, i read on web it said “curl_easy_cookie” function which is introduced on libcurl.

    this feature is missing over php extension, i think libcurl is not liable for this problem rather u should rephrase the following statement :)

    “….But I am saying “fuck cookiejar” because developers of curl didn’t….”

    to

    “….But I am saying “fuck cookiejar” because developers of ‘php curl extension’ didn’t….”

    best wishes :)

  2. well said, hasan, lets point finger to them too!! when I said curl with php, i was actually meant that. maybe just expressed it in a wrong way.

    khik khik. But I am really a big fan of Curl extension. Without it I might have to spend thousands of hours doing it otherwise. he he he.

  3. While developing my Grab Yahoo class, I found out servers running PHP with safe_mode being turned On has this problem. CURLOPT_COOKIEJAR does not write the cookie file.

    The way around that I followed to have a function to set Cookie Jar by using fopen().

    Here is the piece of code:

    function setCookieJar()
    {
    // Sets the encrypted cookie filename using Yahoo! account username
    $this->cookieFilename = MD5($this->login);

    // Sets the Cookie Jar filename with an absolute path
    $this->cookieFileJar = (!empty($this->cookieJarPath)) ? $this->cookieJarPath . “/” . $this->cookieFilename : $_SERVER['DOCUMENT_ROOT'] . “/” . $this->cookieFilename;

    fopen($this->cookieFileJar, “w”);
    }

    Not sure if it is a good way but has solved problem for many who downloaded my class.

  4. Hey, I’m working on a script that does a login to a very fancy ASP site… fancy my ass, they’re fucking idiots but it uses cookies, checks headers, friggin does everything. I was wondering if you could point me in the right direction.

    What I need is to use cUrl to emulate a user using Internet Explorer 5+ with Javascript enabled and login to a website and keep a cookie throughout the entire period while being logged in, if I lose the session cookie I can’t login for another half hour, how fucking gay is that?

    I hope you can help me out a little bit!!

    Any questions or anything contact@johncaza.com !

    Thanks a lot

  5. Rather than bitching about it and disrespecting people you could patch it up. You know, as it has been traditionally done.

    Kids…

  6. I just discovered that one has to perform a transfer, any transfer, to be able to read a cookie file using curl_easy_getinfo. So I set the filename for the cookiefile, set the URL to “”, did the transfer (which fails because of the invalid URL), and then one can access the cookielist via curl_easy_getinfo. Yes, the documentation states this, but what an unintuitive limitation. This was using libcurl 7.18.

  7. Actually, everything you needed to do is add full path to cookie file… i.e:
    /home/user/public_html/junk/cookie.txt

    I bet you’ve added only “./cookie.txt” and it was tryin’ to write it to /cookie.txt … you got the point….

    Yeah, this post is old but still none realized that so far.

    Conclusion: cURL kick ass!

  8. cURL does kick ass and cookiejar in PHP does suck because you CANNOT use CURLOPT_HEADER and CURLOPT_COOKIEJAR/CURLOPT_COOKIEFILE together at the same time for chained requests. You have to either scrap the cookie automation and manage the cookies yourself or scrap the custom headers for all of your requests and accept the defaults provided by cURL. I have yet to see an alternative solution that works.

  9. I had the same problem, only with a twist
    The same script works fine in Mac OS 10.6, cookies are created and stored in a writable directory (not even a full path as someone suggested above), but the script doesn’t work at all in Windows XP Pro

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s