Tuesday, March 17, 2015

Adding info to Laravel logs

I am coding a queue worker that is handling some pretty large (2gig+) datasets and so wanted some details in my logs that Vanilla laravel didn't offer.

Reading the documentation at http://laravel.com/docs/4.2/errors wasn't much help until I twigged that I could manipulate the log object returned by Log::getMonolog();.

Here is an example of adding memory usage to Laravel logs.

In app/start/global.php make the following changes
 Log::useFiles(storage_path().'/logs/laravel.log');  
 $log = Log::getMonolog();  
 $log->pushProcessor(new Monolog\Processor\MemoryUsageProcessor);  

You'll find the Monolog documentation on the repo

Thursday, March 12, 2015

Support for Postgres broken in HHVM 3.6.0

On my desktop machine I run my package upgrades every day.  The other day my Hiphop version got updated to 3.6.0 and suddenly my Postgres support died.

Running Hiphop gave a symbol not found error in the postgres.so file ( undefined symbol: _ZTIN4HPHP11PDOResourceE\n ) exactly like the issue reported on the driver repository (here).

I tried to recompile the postgres driver against Hiphop 3.6.0 but hit a number of problems, mostly to do with hhvm-pgsql-master/pdo_pgsql_statement.cpp it seems.

The fix for the incompatibility was unfortunately rolling back to my previous version of Hiphop.  To do this on Mint/Ubuntu just do this:

  1. Run cat /etc/*-release to get your release information
  2. Download the appropriate package for your distro from http://dl.hhvm.com/ubuntu/pool/main/h/hhvm/
  3. Remove your 3.6.0 installation of hhvm: sudo apt-get remove hhvm
  4. Install the package you downloaded : sudo dpkg -i <deb package>
After that everything should be installed properly and you can start up hhvm without a problem.




Sunday, February 22, 2015

Fixing puppet "Exiting; no certificate found and waitforcert is disabled" error

While debugging and setting up Puppet I am still running the agent and master from CLI in --no-daemonize mode.  I kept getting an error on my agent - ""Exiting; no certificate found and waitforcert is disabled".

The fix was quite simple and a little embarrassing.  Firstly I forgot to run my puppet master with root privileges which meant that it was unable to write incoming certificate requests to disk.  That's the embarrassing part and after I looked at my shell prompt and noticed this issue fixing it was quite simple.

Firstly I got the puppet ssl path by running the command puppet agent --configprint ssldir

Then I removed that directory so that my agent no longer had any certificates or requests.

On my master side I cleaned the old certificate by running puppet cert clean --all (this would remove all my agent certificates but for now I have just the one so its quicker than tagging it).

I started my agent up with the command puppet agent --test which regenerated the certificate and sent the request to my puppet master.  Because my puppet master was now running with root privileges (*cough*) it was able to write to its ssl directory and store the request.

I could then sign the request on my puppet master by running puppet cert sign --all

When running normally the puppet master will run as the puppet user so I'm not overly worried about running it as root in CLI while I debug it.

Thursday, February 19, 2015

Installing the Postgres extension for Hiphop

PostGres is rapidly gaining traction as a solid relational database manager.  It provides transaction reliability (ACID), standards compliance, and has a better reputation for handling large datasets than MariaDB / mySQL.

Luckily installing it for Hiphop is painless and there is an externally provided extension for it.

There was not a prebuilt package available for the PocketRent/hhvm-pgsql extension for my version of Hiphop so I built it following the advice in the project readme.  Their instructions worked first time.

I did not have success with using the "ini" method of including the extension so had to create a Hiphop configuration file.

 DynamicExtensionPath = /path/to/hhvm/extensions  
 DynamicExtensions {  
   * = pgsql.so  
 }  

I placed the snippet they provide (above) into a file called config.hdf in my Hiphop location.  I'm using Mint on my dev box so this was /etc/hhvm/config.hdf for me.

Then I edited /etc/default/hhvm and set Hiphop up to use the config file.  This snippet shows the change:

 ## Add additional arguments to the hhvm service start up that you can't put in CONFIG_FILE for some reason.  
 ## Default: ""  
 ## Examples:  
 ##  "-vLog.Level=Debug"        Enable debug log level  
 ##  "-vServer.DefaultDocument=app.php" Change the default document  
 ADDITIONAL_ARGS="-c /etc/hhvm/config.hdf"  

After that I restarted my hhvm service and was able to use Postgres in my Laravel project.  Hurray :)

New site in nginx is downloading PHP instead of executing it

I've just set up a new nginx host and was having problems with visiting it.

Whenever I loaded the page it would download a copy of my index file.

In my case the problem was two-fold.  Firstly I had mistyped my server name so it was falling back to an alternative catchall.

Secondly Chromium has a "won't fix" bug whereby redirects are cached.  Interestingly the cache also persisted into Incognito mode.

Clearing my cache with the "Clear Browsing History" menu function didn't work to clear out the redirect cache.  This gave the symptom of stopping nginx, stopping varnish, and even stopping hhvm but still having Chromium download the file when I tried to visit it.

What did work was to affix a get variable to the link.  So instead of visiting http://mysite.local I tried http://mysite.local?foo=1 which invalidated the cache.

This wasn't an entirely satisfactory solution so the next thing I tried was to use the developer tools (ctrl shift I) and then right clicking on the file in the network log.  Using the "Clear Browser Cache" option from that popup also worked.

So the TL;DR is:
1) Make sure that your config file is correct
2) Clear out your redirect cache or invalidate it

Wednesday, January 7, 2015

Compressing Apache output with mod_deflate on Centos

Apache on Centos ships with mod_deflate installed and enabled by default.  To check this you can grep your config file and make sure the line which loads it is not commented out.

 cat /etc/httpd/conf/httpd.conf | grep LoadModule deflate_module  

When Apache loads it reads all the config files (ending in .conf) in /etc/httpd/conf.d so we'll add configuration options for mod_deflate into this directory. Lets use a file called deflate.conf to specify the config:

 <IfModule mod_deflate.c>  
  AddOutputFilterByType DEFLATE text/plain  
  AddOutputFilterByType DEFLATE text/html  
  AddOutputFilterByType DEFLATE text/xml  
  AddOutputFilterByType DEFLATE text/css  
  AddOutputFilterByType DEFLATE text/javascript  
  AddOutputFilterByType DEFLATE image/svg+xml  
  AddOutputFilterByType DEFLATE image/x-icon  
  AddOutputFilterByType DEFLATE application/xml  
  AddOutputFilterByType DEFLATE application/xhtml+xml  
  AddOutputFilterByType DEFLATE application/rss+xml  
  AddOutputFilterByType DEFLATE application/javascript  
  AddOutputFilterByType DEFLATE application/x-javascript  
  DeflateCompressionLevel 9  
 # Browser specific settings  
  BrowserMatch ^Mozilla/4 gzip-only-text/html  
  BrowserMatch ^Mozilla/4\.0[678] no-gzip  
  BrowserMatch \bMSIE !no-gzip !gzip-only-text/html  
  BrowserMatch \bOpera !no-gzip   
 </IfModule>  

You can check it is working by noticing your YSlow report now shows, by using an online tool, or by just checking the headers with Chrome or Firefox's developer tools.

If you're using Varnish and Apache does not have mod_deflate then you can enable gzip in your vcl as per the Varnish manual.  The page linked at the bottom of the Varnish manual ( How GZIP, and GZIP+ESI works in Varnish ) explains how the response from the backend is stored in a compressed state.

  sub vcl_fetch {  
    if (beresp.http.content-type ~ "text") {  
        set beresp.do_gzip = true;  
    }  
 }  

Tuesday, January 6, 2015

Separating business logic from persistence layer in Laravel

There are several reasons to separate business logic from your persistence layer.  Perhaps the biggest advantage is that the parts of your application which are unique are not coupled to how data are persisted.  This makes the code easier to port and maintain.

I'm going to use Doctrine to replace the Eloquent ORM in Laravel.  A thorough comparison of the patterns is available here.

By using Doctrine I am also hoping to mitigate the risk of a major version upgrade on the underlying framework.  It can be expected for the ORM to change between major versions of a framework and upgrading to a new release can be quite costly.

Another advantage to this approach is to limit the access that objects have to the database.  Unless a developer is aware of the business rules in place on an Eloquent model there is a chance they will mistakenly ignore them by calling the ActiveRecord save method directly.

I'm not implementing the repository pattern in all its glory in this demo.  For a more purist approach to the pattern you can read this.  The reason that I'm choosing this approach is to cut down on the number of classes, cut down on the composer autoload, and to ensure my code is portable.  I also do not have to make any changes to my app aliases in config or create new Laravel services.

I'm going to use three objects to refer to my user table:
  • The UserEntity is a Doctrine entity to be accessed via the Doctrine entity manager by the Repository.  
  • The UserRepository is an intermediary layer akin to the data access object of other languages.  It uses the Entity to gather information that the Service layer needs.  
  • The UserService implements business logic and exposes methods to the controller.  It is the "fat model" in the "fat model / skinny controller" paradigm.
In order to be able to use Doctrine within Laravel I'm using the mitchellvanw/laravel-doctrine package.  I also use "raveren/kint" for access to the debugging "dd" shortcut.

The decision to use constructor injection when instantiating the user service is to make it easier to use a mock object when testing.  

I've deviated from the common practice of using private properties on a Doctrine entity and rather exposing getters and setters.  This is primarily so that in my user service layer I can conveniently reference properties of the entity in a manner that is not likely to change if I swap to another ORM.

I decided against marking the entity private and using reflection to retrieve the private properties in my repository.  I felt it was an unnecessary complication and not worth the processing cycles to ensure compatibility between methods of accessing a model property between ORMs.

The Doctrine entity class is incapable of persisting itself so if a developer instantiates it and modifies properties in the controller they won't be able to persist it unless they call the EntityManager class.  Hopefully this is more PT than calling save() on an ActiveRecord object.  Our design philosophy of avoiding doing this in controllers should also help to discourage mistakes here.

The files I created are listed below.  The drawback of avoiding any Laravel specific code is the rather ugly way of instantiating the service in the controller.  I believe, however, that my code will be easier to port to another framework than if I were to declare a Laravel service provider to make a static call to user.

You must include "app/models/user" into your composer autoload section and run the composer dump-autoload command from your shell once you've set up the directory.

app/models/user/UserEntity.php

 namespace User;  
 use Doctrine\ORM\Mapping AS ORM;  
 /**  
  * @ORM\Entity  
  * @ORM\Table(name="users")  
  */  
 class UserEntity  
 {  
      // see http://docs.doctrine-project.org/projects/doctrine-orm/en/latest/reference/basic-mapping.html for info on mapping  
      /**  
       * @ORM\Id  
       * @ORM\GeneratedValue  
       * @ORM\Column(type="integer")  
       */  
      private $id;  
      /**  
       * @ORM\Column(type="string")  
       */  
      private $name;  
      /**  
       * @ORM\Column(type="string")  
       */  
      private $password;  
      /**  
       * @ORM\Column(type="datetime")  
       */  
      private $created;  
      /**  
       * @ORM\Column(type="datetime")  
       */  
      private $modified;  
      public function getId()  
      {  
           return $this->id;  
      }  
      public function getName()  
      {  
           return $this->name;  
      }  
      public function setName($name)  
      {  
           $this->name = $name;  
      }  
      public function setPassword($password)  
      {  
           $this->password = $password;  
      }  
 }  

app/models/user/UserRepository.php

 <?php namespace User;  
 class UserRepository  
 {  
   protected $entity;  
   public function __construct( UserEntity $userEntity )  
   {  
     $this->entity = $userEntity;  
   }  
   public function getUserById($userId)  
   {  
     $user = \EntityManager::find('User\UserEntity', $userId);  
     return $user;  
   }  
   public function getUserByName($userName)  
   {  
     $user = \EntityManager::getRepository( 'User\UserEntity' )->findBy( [ 'name' => $userName ] );  
     if( !is_array( $user ) || empty( $user ) )  
     {  
       return false;  
     }  
     return $user[0];  
   }  
   public function setPassword( $userDetails, $password )  
   {  
     // If user variable is numeric, assume ID  
     if ( is_numeric( $userDetails ) )  
     {  
       // Get user based on ID  
       $user = $this->getuserById( $userDetails );  
     }  
     else  
     {  
       // Since not numeric, lets try get the user based on Name  
       $user = $this->getuserByName( $userDetails );  
     }  
     $user->setPassword( $password );  
     $this->persist( $user );  
     return true;  
   }  
   public function persist( UserEntity $user )  
   {  
     // do any last moment validations here  
     \EntityManager::persist( $user );  
     \EntityManager::flush();  
   }  
 }  

app/models/user/UserService.php

 <?php namespace User;  
 /**  
  * Our UserService, containing all useful methods for business logic around Users  
  * Do not reference the entity in here.  
  */  
 class UserService  
 {  
   // Containing our user repository to make all our database calls to  
   protected $userRepo;  
   /**  
    * Loads our $userRepo with the supplied userRepository  
    *  
    * We use constructor injection to make it easier to unit test.  
    *  
    * @param userInterface $userRepo  
    * @return userService  
    */  
   public function __construct( $userRepository )  
   {  
     $this->userRepo = $userRepository;  
   }  
   /**  
    * Method to get user based either on name or ID  
    *  
    * @param mixed $user  
    * @return string  
    */  
   public function getUserName($user)  
   {  
     // If user variable is numeric, assume ID  
     if (is_numeric($user))  
     {  
       // Get user based on ID  
       $user = $this->userRepo->getuserById($user);  
     }  
     else  
     {  
       // Since not numeric, lets try get the user based on Name  
       $user = $this->userRepo->getuserByName($user);  
     }  
     // If user entity returned (rather than null) return the name of the user  
     if ($user != null)  
     {  
       return $user->getName();  
     }  
     // If nothing found, return this simple string  
     return 'user Not Found';  
   }  
   public function setPassword( $user, $password )  
   {  
     // perform any validations  
     if( strlen( $password ) < 6 )  
     {  
       throw new \ValidationException( 'Password may not be shorter than 6 characters' );  
     }  
     // perform any hashing on the password  
     $password = \Hash::make($password);  
     return $this->userRepo->setPassword( $user, $password );  
   }  
 }  

app/controllers/HomeController.php

 <?php  
 class HomeController extends BaseController {  
      public function showWelcome()  
      {  
           return View::make('hello');  
      }  
      public function setPassword( $userDetails, $password )  
      {  
           $userService = new User\UserService( new User\UserRepository( new User\UserEntity ) );  
           try  
           {  
                $response = $userService->setPassword( $userDetails, $password );  
           }  
           catch ( ValidationException $e )  
           {  
                // set error message for frontend  
                echo 'An exception was thrown ('.$e->getMessage().') - this will result in a frontend message';  
           }  
           dd( $response );  
      }  
 }  

app/routes.php

 Route::put('/password/{user}/{password}', 'HomeController@setPassword');