June 23, 2015

MarkDown to PDF ::: creating pdf e-book from series of markdown

I was looking for an easy way to read the 'famous' series of articles of Fabien Potencier (https://github.com/fabpot/Create-Your-Framework). They are series of blog posts of him and later stored at github as *.rst aka markdown.


I thought and searched for quite a bit for some tools to get them all into single e-book like pdf e-book because I became lazy enough to retrieve them from bookmark every time I need them.

I came across http://engineeredweb.com/blog/2014/convert-markdown-pdf-using-php.  An wonderfully written tutorial for similar purposes. While looking into the code - I remember an old `codelet` of mine which actually can solve lots of similar problems. It was using 'KnpSnappy' bundle.


So I got some code copied from Matt Farina from above mentioned blog post and modified my codeblock a bit -- which managed to build up a solution for my problem.

I will go step by step for my own record:

(1) Bootstrapping:


a) Lets create a new project at our webroot. Let's name it "MarkDownToPdf".
b) Now inside the project let's create the files we will require:

himel in /var/www/MarkDownToPdf$ touch index.php autoload.php composer.json .gitignore

c) Now let's open the project with a suitable IDE (or one can use any tools at comfort) - in my case I am using PHPStorm.

d) After that we will edit the
composer.json
with our required libraries (like as follows):

{
  "require" :  {
    "neurosys/file-merger": "dev-master",
    "h4cc/wkhtmltopdf-i386": "0.12.x",
    "h4cc/wkhtmltopdf-amd64": "0.12.x",
    "knplabs/knp-snappy": "0.3.*@dev",
    "dompdf/dompdf": "0.6.*",
    "michelf/php-markdown": "1.4.*",
    "querypath/querypath": "3.*",
    "masterminds/html5": "1.*"
  }
}

e) Now it's time for running the composer update. But before that we need to have composer in our system. To download composer we will run the following:
himel in /var/www/MarkDownToPdf$ curl -sS https://getcomposer.org/installer | php
Then run composer:
himel in /var/www/MarkDownToPdf$ php composer.phar install
All the libraries will be at the "vendor" folder.

f) Now as the vendors are downloaded it will be better to create the ".gitignore" file to avoid future problems when we will be adding this to git. My ".gitignore" contains:

/vendor/
/composer.phar
/composer.lock
/.idea

g) Now it's time to write the ``autoload.php``. It will be used to autoload the vendors - it puts a direct require to the autoload file inside the Vendor folder.
<?php

require_once __DIR__ . '/vendor/autoload.php';


(2) Downloading the files:

Now I will download the files and for this I will create a ``files`` folder. Point to be noted that we can use any files at any location but to make it a complete solution - I am keeping the files inside the tiny app.
himel in /var/www/MarkDownToPdf$ mkdir files
himel in /var/www/MarkDownToPdf$ cd files
himel in /var/www/MarkDownToPdf/files$ git clone git@github.com:fabpot/Create-Your-Framework.git fabpot

So the books are downloaded at "/fabpot/book" inside "files" folder.

(3) The Code:


The code will first include the required libraries into it. So let's start like:
<?php

require_once __DIR__ . '/autoload.php';

use NeuroSys\FileMerger\Merger;
use NeuroSys\FileMerger\Driver\PdfTkDriver;
use NeuroSys\FileMerger\Transformer\ImageTransformer;
use Knp\Snappy\Pdf;


Now I will take the path of the folder where the markdown files are in - as input from command line:
$handle = fopen("php://stdin","r");
echo "Path to markdown files (upto folder without trailing slash '/' ):" . PHP_EOL;
$dir = rtrim(fgets($handle, 1024));
fclose($handle);

Now as we do have the folder path - we can read all the files in there.
$files = scandir($dir);

So let's process the files one by one
$pdfs = array();
$domain_name = 'http://fabien.potencier.org/';

$index = 1;
foreach ($files as $file){
    if (!in_array($file, array('.', '..'))) { // as scandir() returns file list with dots which actually specifies parent directory
        
    }
}

Inside the loop I will convert markdown files to html first:
// md to html
$markdown = file_get_contents( $dir.'/'.$file );
$markdownParser = new \Michelf\MarkdownExtra();
$html = $markdownParser->transform( $markdown );

For the output html we need to do some cleanup and modify links in the html if there's any:
// html clean up and clean up links by assigning absolute path to them
$dom = \HTML5::loadHTML( $html );
$links = htmlqp( $dom, 'a' );

foreach ( $links as $link ) {
    $href = $link->attr( 'href' );
    if ( substr( $href, 0, 1 ) == '/' && substr( $href, 1, 1 ) != '/' ) {
 $link->attr( 'href', $domain_name . $href );
    }
}

$html = \HTML5::saveHTML( $dom );

Now to initiate KnpSnappy Bundle and its PDF feature - we need to install "wkhtmltopdf". We can do this by:
himel in /var/www/MarkDownToPdf$ sudo apt-get install wkhtmltopdf
or
himel in /var/www/MarkDownToPdf$ sudo apt-get install xfonts-base xfonts-75dpi
himel in /var/www/MarkDownToPdf$ wget http://sourceforge.net/projects/wkhtmltopdf/files/0.12.2.1/wkhtmltox-0.12.2.1_linux-wheezy-amd64.deb
himel in /var/www/MarkDownToPdf$ sudo dpkg -i wkhtmltox-0.12.2.1_linux-wheezy-amd64.deb

Then we will use this bundle to convert html to pdf. We are saving the output pdf file in "pdf" folder inside the given path.
We will store the path of the pdf file in an array which will help us combining them.
// html to pdf
$snappy = new Pdf( '/usr/bin/wkhtmltopdf' );
$snappy->generateFromHtml( $html, $dir . '/pdf/' . $index . '.pdf' );
$pdfs[$index] = $dir . '/pdf/' . $index . '.pdf';


We have to install "pdftk" for the merging task.
himel in /var/www/MarkDownToPdf$ sudo apt-get install pdftk

Now outside the loop we will iterate over the pdf file path array and combine them and give the final path as output:
// merging all files
$driver = new PdfTkDriver("/usr/bin/pdftk");
$merger = new Merger($driver);
$merger->addTransformer(new ImageTransformer($snappy));

foreach ($pdfs as $pdf) {
   $merger->addFile($pdf);
}

$merger->merge($dir . '/pdf/CreateYourOwnFrameWork-FabPot.pdf');
echo PHP_EOL;
echo "Combined e-book is: " . $dir . '/pdf/CreateYourOwnFrameWork-FabPot.pdf';

(4) Running:


Now we will run our file:
himel in /var/www/MarkDownToPdf$ php index.php 
Path to markdown files (upto folder without trailing slash '/' ):

We will input the path of the folder as: /var/www/MarkDownToPdf/files/fabpot/book

(5) The Output:


The output is:
Combined e-book is: /var/www/MarkDownToPdf/files/fabpot/book/pdf/CreateYourOwnFrameWork-FabPot.pdf


The book is under Creative Common License so this permits me sharing the final output.

Slideshare link of the final output is:


The codebase is in github (https://github.com/himelnagrana/MarkDownToPdf)


Thanks.
And a thousand thanks to Matt Farina.

May 17, 2010

Experiencing Facebook Javascript SDK with graph API - II

Previous post:

http://nagbaba.blogspot.com/2010/05/experiencing-facebook-javascript-sdk.html 

Here goes the link of the first part. In this post I will share (or better to say note down about further experience in FB Javascript API).

Here I will try to focus on :::

(2) get fans count and their feed on page's wall to promote and to share the page.

 (3) post to wall (to my page or profile) from my site. 

 

Steps for task (2) :

[1] Facebook fan pages are awesome tools for marketing or promotion. They are attached to facebook open graph. SO it is very much easy to access their public info like feed stream and fan count. But for the other info there should be nneded to logging in.

[2] If we hit to https://graph . facebook. com/ we will get all the public info like -- id, name, username(if any), page profile picture, link, location, hours open. For other info we have to use metdata tag. for this purpose we may hit https://graph . facebook. com/ ? metadata= 1. We may find links for feed, posts, tagged, statuses, links, notes, photos, albums, events and videos. Among them feed, posts, photos and albums are publicly accessible.

[3] So using this protocol we can find and fulfil our needs as mentioned above.

[4] We have called javascript api for getting fancount and for feeds we have used php as this is easy to maintain a multilevel data structure like array in php than javascript.

[5] So the frame of our code goes like as below :


<!doctype html>
<html>
  <head>
    <meta http-equiv="content-type" content="text/html; charset=utf-8"/>
  </head>
  <body>
    <div id="user-info" style="display: none;"></div>
    <script type="text/javascript" src="http://ajax.googleapis.com/ajax/libs/jquery/1.3.2/jquery.min.js"></script>
    <div id="fb-root"></div>
    <script type="text/javascript" src="http://connect.facebook.net/en_US/all.js"></script>

    <script type="text/javascript">
            $(document).ready(function(){
                FB.init({ apiKey: 'XXXXXXXXXXXXXXXXXXX' });
            });
    </script>
 </body>
</html>

[6] Here the FB.init is used to initiate the api call. Now to get the fan count we will add the following function after document.ready block and call the function from document.ready.

function getUpdate () {
var htmls = '<div style="background-color:#77A1CD:color:#EAAA00;">';

FB.api('/224365082363', function(response1) {
                        total_members = response1.fan_count;
                        name = response1.name;
                        link = response1.link;
                        htmls += 'Total <i>'+total_members+'</i> persons likes '+'<a href="'+link+'">'+name+'</a><br>';
                        htmls += '</div>';
                        $('#user-info').show();
                        $('#user-info').attr("style","width:450px;height:auto;background-color:#E4E9EE");
                        $('#user-info').html(htmls);
               });
}

[7] Now to find the lates feeds we have used php. From php we have called the same graph protocol and used file_get_contents. the returned datatype is json. SO we used json_decode to parse the data. Then the data type becomes an stdclassobject. The code goes as follows:

<?php
 $myObj = json_decode(file_get_contents("https://graph.facebook.com/224365082363/feed"));
echo "<br>";
$i = 0;
        foreach ($myObj->data as $aData){
                $user[$i]['from'] = $from = "<a href=\"http://www.facebook.com/profile.php?id=".$aData->from->id."\" >".$aData->from->name."</a>";
                $user[$i]['msg'] = $message = $aData->message;
                $created = $aData->created_time;
                
                
                $html  = "<div style=\"background-color:#F9F9F9;color:#000000;height:auto;width:500px;\">";
                $html .= $from.": ".$message;
                $html .= "<font size=\"1\">   ".$created."</font>";
                $html .= "<hr style=\"border-style:dotted; border-color:#6D84B4\"/>";
                $html .= "</div>";
                echo $html;
                $i++;
        }
?> 

[8] Here the time returned is on 'yyyy-mm-ddThh:mm:ss' format but normally and specially our application demands time difference from now liek as how many mniutes ago the feed was posted. So for these reason we have used a function that calculates the time difference in days, hours and miniutes. the code block for the function and the calling will be as follows:

<?php
 $myObj = json_decode(file_get_contents("https://graph.facebook.com/224365082363/feed"));
echo "<br>";
$i = 0;
        foreach ($myObj->data as $aData){
                $user[$i]['from'] = $from = "<a href=\"http://www.facebook.com/profile.php?id=".$aData->from->id."\" >".$aData->from->name."</a>";
                $user[$i]['msg'] = $message = $aData->message;
                $created = $aData->created_time;
                $createdtime = date('Y-m-d h:i:s', strtotime($created));
                $now = date('Y-m-d h:i:s', time());
                $user[$i]['ago'] = $difference = get_time_difference($createdtime, $now);
                
                $html  = "<div style=\"background-color:#F9F9F9;color:#000000;height:auto;width:500px;\">";
                $html .= $from.": ".$message;
                $html .= "<font size=\"1\">   ".$created."</font>";
                $html .= "<hr style=\"border-style:dotted; border-color:#6D84B4\"/>";
                $html .= "</div>";
                echo $html;
                $i++;
        }

function get_time_difference($start, $end){

        $tempstart1 = explode('-',$start);
        $startyr = $tempstart1[0];
        $startmon = $tempstart1[1];
        $tempstart2 = explode(' ',$tempstart1[2]);
        $startday = $tempstart2[0];
        $tempstart3 = explode(':',$tempstart2[1]);
        $starthr = $tempstart3[0];
        $startmin = $tempstart3[1];
        $startsec = $tempstart3[2];

        $tempend1 = explode('-',$end);
        $endyr = $tempend1[0];
        $endmon = $tempend1[1];
        $tempend2 = explode(' ',$tempend1[2]);
        $endday = $tempend2[0];
        $tempend3 = explode(':',$tempend2[1]);
        $endhr = $tempend3[0];
        $endmin = $tempend3[1];
        $endsec = $tempend3[2];
        

        $start1 = mktime($starthr, $startmin, $startsec, $startmon, $startday, $startyr);
        $end1 = mktime($endhr, $endmin, $endsec, $endmon, $endday, $endyr);

        $dateDiff = $end1 - $start1;
        $fullDays = floor($dateDiff/(60*60*24));
        $fullHours = floor(($dateDiff-($fullDays*60*60*24))/(60*60));
        $fullMinutes = floor(($dateDiff-($fullDays*60*60*24)-($fullHours*60*60))/60);
        
        $ret = "";
        if($fullDays > 0){
                $ret .= "$fullDays Days ";
        }
        if($fullHours > 0){
                $ret .= "$fullHours Hours ";
        }
        if($fullMinutes > 0){
                $ret .= "$fullMinutes Minutes ";
        }

        if($ret!= ''){
                $ret .= "ago";
        } else {
                $ret .= "A Few Moments ago";
        }

        return $ret;
}
?> 

[9] so the final out put will be as the following screenshot -


So we will be back soon with posting to wall steps. ;-)

May 11, 2010

Experiencing Facebook Javascript SDK with graph API - I

This is the first time I am using Facebook Javascript API. In fact this is the first time I am using any facebook api. So I am excited to some extend. I have heard that the api and documentations are easy enough to understand. I read the facebook release of graph api and f8 and they looked coooooool to work with.

The basic purpose of my work was three things -
(1) Get Home page feeds and present them as scrolling in my site
(2) create custom buttons with friends and/or fans count to promote
(3) post to wall (to my page or profile) from my site.

To the experts these are definitely piece-of-cake but for a first timer like me they are the basic things to work with.

So in this post I will discuss  how i learned to get the home page feeds.

Steps for (1) Get Home page feeds and present them as scrolling in my site:

[a] I have created an canvas application called testfbapp (this is just to get an api key) from http://developers.facebook.com/setup . I have set my site name as testfbapp and my site url as http://localhost/testfbapp (as i am planning to test my codes on localhost).

[b] Now from the next page that comes after creating the application I have clicked the developer dashboard link. There all the application that just have been created will be shown. I have copied the API Key, the secret key and the application ID for the app and kept that somewhere.

[c] Then I have searched for Javascript SDK and found it on http://github.com/facebook/connect-js/ . I have downloaded the sdk and copied that to my local directory which is testfbapp under my web root.


[d] In order to get used to it i have roamed around the files I have downloaded and found that it the sdk supports a lot variety of popular javascript libraries like dojo, jquery, mootools, prototype, yui2. Because of previous little experience in jquery I have chosen that.

[e] Now I have created a file named index.php at the project root. I have copied contents from the login.html from examples/jquery directory and pasted it into the index.php page.

[f] Then I have removed the contents of handleSessionResponse() after the following code block

if (!response.session) {
          clearDisplay();
          return;
        }

So the page looks like as follows:
<!doctype html>
<html>
  <head>
    <meta http-equiv="content-type" content="text/html; charset=utf-8"/>
  </head>
  <body>
    <div>
      <button id="login">Login</button>
      <button id="logout">Logout</button>
      <button id="disconnect">Disconnect</button>
    </div>
    <div id="user-info" style="display: none;"></div>

    <script type="text/javascript" src="http://ajax.googleapis.com/ajax/libs/jquery/1.3.2/jquery.min.js"></script>

    <div id="fb-root"></div>
    <script type="text/javascript" src="http://connect.facebook.net/en_US/all.js"></script>

    <script type="text/javascript">
      // initialize the library with the API key
      FB.init({ apiKey: '.....' });

      // fetch the status on load
      FB.getLoginStatus(handleSessionResponse);

      $('#login').bind('click', function() {
        FB.login(handleSessionResponse);
      });

      $('#logout').bind('click', function() {
        FB.logout(handleSessionResponse);
      });

      $('#disconnect').bind('click', function() {
        FB.api({ method: 'Auth.revokeAuthorization' }, function(response) {
          clearDisplay();
        });
      });

      // no user, clear display
      function clearDisplay() {
        $('#user-info').hide('fast');
      }

      // handle a session response from any of the auth related calls
      function handleSessionResponse(response) {
        // if we dont have a session, just hide the user info
        if (!response.session) {
          clearDisplay();
          return;
        }
      }
    </script>
  </body>
</html>

[g] Now I have read about the graph api of facebook from http://developers.facebook.com/docs/api and for our purpose (i.e. to get home page feeds) I found that
https://graph.facebook.com/user-name/home link is helpful. So I have tested on the browser address bar putting my FB user name and some of my friends FB user name and found that it works only for current user that the logged in user. but https://graph.facebook.com/user-name works for self and friends information. It fetches only publicly viwable information. So our next step should bego through the javascript api reference to utilize the graph api.

[h] I have gone through http://developers.facebook.com/docs/reference/javascript and found that FB.api  API call is the right method to use as it can directly call to graph API.

[i] I have put my API key for my application in the FB.init() method and following this guide I have written the following code in the handleSessionResponse() method. I have checked https://graph.facebook.com/user-name?metadata=1 url in my browser to find the proper link for home feeds. I have used

for(property in response){
   alert(property);
}

to check the properties of the response object. And then for output I have written --

var user_id = FB.getSession().uid;
FB.api('/'+user_id+'/home', function(response) {
  var total_feed = response.data.length;
   var htmls = '<ul style="list-style-image: none; list-style-position: outside; list-style-type: none;">';       
 for(var x = 0; x < total_feed; x++) {       
  name = response.data[x].from.name;       
  id = response.data[x].from.id;       
  message = response.data[x].message;       
  created_time = response.data[x].created_time;              
  htmls += ""; 
  htmls += '<a href="http://www.facebook.com/profile .php ?id='+id+'>'+  name+'</a>:';
  htmls += message;
  htmls += " created at ::: "+created_time+""; 
 htmls += "";   
}      
htmls += "</ul>";
  $('#user-info').show();
  $('#user-info').attr("style","width:450px;height:auto;background-color:#E4E9EE");
  $('#user-info').html(htmls);
});


[j] Here FB.getSession() holds all the session variables after authentication. If some user visiting my page is not logged in the FB.getSession() is null and it will not return any feed. To login there is a login button at the top of the page (at the top of my code). After clicking that button there will be an iFrame dialogue box appeared and will ask permission from the user to login and then to allow the application to fetch  profile data.

[k] Now if we run http://localhost/testfbapp and login using FB credential and allow this application we will see the recent homepage feeds.

[l] Now to automate this script that is to get live updates we may modify the codes like as follows and this is our final version of this code.

<html>
  <head>
    <meta http-equiv="content-type" content="text/html; charset=utf-8"/>
  </head>
  <body>
    <div>
      <button id="login">Login</button>
      <button id="logout">Logout</button>
      <button id="disconnect">Disconnect</button>
    </div>
    <div id="user-info" style="display: none;"></div>

    <script type="text/javascript" src="http://ajax.googleapis.com/ajax/libs/jquery/1.3.2/jquery.min.js"></script>

    <div id="fb-root"></div>
    <script type="text/javascript" src="http://connect.facebook.net/en_US/all.js"></script>

    <script type="text/javascript">
      // initialize the library with the API key
      FB.init({ apiKey: 'XXXXXXXXXXXX' });

      // fetch the status on load
      FB.getLoginStatus(handleSessionResponse);

      $('#login').bind('click', function() {
        FB.login(handleSessionResponse);
      });

      $('#logout').bind('click', function() {
        FB.logout(handleSessionResponse);
      });

      $('#disconnect').bind('click', function() {
        FB.api({ method: 'Auth.revokeAuthorization' }, function(response) {
          clearDisplay();
        });
      });

      // no user, clear display
      function clearDisplay() {
        $('#user-info').hide('fast');
      }

      // handle a session response from any of the auth related calls
      function handleSessionResponse(response) {
        // if we dont have a session, just hide the user info
        if (!response.session) {
          clearDisplay();
          return;

        var user_id = FB.getSession().uid;
        var access_token = FB.getSession().access_token;

        getUpdate();

      }

      function getUpdate () {
              var user_id = FB.getSession().uid;
              //alert(user_id);
              var access_token = FB.getSession().access_token;
              //alert(access_token);
              FB.api('/'+user_id+'/friends', function(response) {
                 var total_friend = response.data.length;
                 //alert(total_friend);

              });
              FB.api('/'+user_id+'/home', function(response) {
                        var total_feed = response.data.length;
                        var htmls = '<ul style="list-style-image: none; list-style-position: outside; list-style-type: none;">';                         
for(var x = 0; x < total_feed; x++) {        
name = response.data[x].from.name;        
id = response.data[x].from.id;        
message = response.data[x].message;        
created_time = response.data[x].created_time;               
htmls += "";  htmls += '<a href="http://www.facebook.com/profile .php ?id='+id+'>'+ name+'</a>:';
htmls += message;
htmls += " created at ::: "+created_time+""; 
htmls += "";   
}
                        htmls += "";
                        $('#user-info').show();
                        $('#user-info').attr("style","width:450px;height:auto;background-color:#E4E9EE");
                        $('#user-info').html(htmls);
                });
               

                t = setTimeout(getUpdate,15000);
        }

        

     </script>
  </body>
</html> 


Now this will be the ned of part 1 getting FB homepage feed.
[cont...]