Friday, 11 June 2010

Conversion of UTF-8 form fields in Multi Part Form

Got to say a big thanks to Abhishek at Running Commentary for the solution to this problem.

While using the Apache Commons File Upload Library on a Spanish Language form, I was having problems with the UTF-8 characters that were being submitted in people's names etc. Everything else was working properly, the rest of the Spanish characters on the page were displaying properly but the input was being translated into garbage when submitted.

The solution ultimately was to change

item.getString()

to

item.getString("UTF-8").trim()

While extracting the items from the List element that the FileUploadHandler returned after parsing the request object.

Bear in mind, it doesn't matter what you do before calling this, if you don't explicitely tell the FUH what encoding you want to you use, you'll always get standard ASCII back (and subsequently garbage if your form fields contain anything other than standard ASCII...

Hope this helps.

Friday, 23 April 2010

How to ask questions the smart way

Found this absolutely fantastic article on how to ask questions on forums, newsgroups etc. in an intelligent and considered way that is more likely to get you a decent response.

Very good advice... http://www.catb.org/~esr/faqs/smart-questions.html

How to change the default JRE in Mac OS X (Leopard)

The problem was, my Eclipse installation was setup to use JRE 1.6 but when I checked using
mvn --version
it reported that it was using JRE 1.5. The solution's pretty simple in fact. The versions of Java JRE that are installed on your Mac are located in
/System/Library/Frameworks/JavaVM.framework/Versions
If you list the contents of the folder in terminal using
ls -al
then you'll see a symbolic link called "CurrentJDK" pointing to a symbolic link called "1.5" pointing to the latest version of 1.5 installed. Delete this link and recreate it pointing it to the symbolic link called "1.6" and you're done. The following code will achieve this.
 cd /System/Library/Frameworks/JavaVM.framwork/Versions/
sudo rm -r CurrentJDK
sudo ln -s 1.6 CurrentJDK
UPDATE Due to obvious confusion caused by my $ sign, I have reworked the code snippets to make them clearer

Learning Java now :)

So, the company where I am working at the moment has made the move into a Java based web environment. Lots of reasons and I'm not going into them here as this is a personal blog (despite the technical nature) and I'm not going to discuss work decisions or anything that may be confidential in nature of course.

But - I have a blog, I'm learning a lot of new stuff and it makes sense to switch the focus of the blog to whatever I'm using/doing/learning rather than a single focus on Symfony as I originally planned.

So watch out for a lot of new posts as I go... most of what I learn will posted here for my future reference

Saturday, 6 February 2010

Quick and easy server control

I use a macports installation of Apache2, PHP5 and MySQl5 for development on my macbooks. This is a great system and really simplifies the installation and setup the utilities.

However, the start/stop commands for each are not added to the PATH by default and it's tough remembering the long paths to the control programs.

But there's a solution:

  1. Open the Terminal
  2. Make sure that you're in your home folder.
  3. Edit (or create) .bash_profile


    $ nano .bash_profile
  4. Add aliases for your commands


    alias apache2ctl="sudo /opt/local/apache/bin/apachectl"
    alias mysqlstart="sudo /opt/local/bin/mysqld_safe5"
    alias mysqlstop="sudo /opt/local/bin/mysqladmin5 -u root -p shutdown"
  5. restart your bash session with


    source ~/.bash_profile
    and you're good to go...
Then you simply issue the following commands from the command line.
apache2ctl start # Start Apache
apache2ctl stop # Stop Apache
apache2ctl graceful # Restart Apache
mysqlstart # Start Mysql
mysqlstop # Stop Mysql

Wednesday, 3 February 2010

Organising Partials in Symfony

A little tip for organising partial templates in Symfony.

If you want to store partials in sub folders, simply name the first level of folders prefixed with an underscore and remove the underscore from the partial name (or make sure include the underscore when you're calling it.

That way, let's say you send out emails for various reasons on your site, so you have lots of templates (partials) in a mailer module to make your emails nice and easy to manage. If you wanted to organise your registration email body and subject partials in sub folders, you could do the following:

Your subject partial would be located at:

    /apps/frontend/modules/mailer/templates/_subjects/registration.php 

and your body partials might be located at:

    /apps/frontend/modules/mailer/templates/_registration/body_html.php
    /apps/frontend/modules/mailer/templates/_registration/body_text.php

You would then call these partials from your sendMail action as follows:

    // subject 
    include_partial('mailer/subjects/registration');

    // email body
    $message->setBody($this->getPartial('mailer/registration/body_html'), 'text/html')
        ->addPart($this->getPartial('mailer/registration/body_text'), 'text/plain');

Thanks to my colleague Eric for helping work this one out. A good idea that we'll use a lot now to help organise our templates and partials in all sorts of wonderful ways

Monday, 1 February 2010

Adding embedded forms dynamically using ajax

Over this last weekend, I've been building an 'invite your friends' form. Fairly standard fair, a registered user is presented with the opportunity to send invitations to friends for one reason or another by entering their name & email address. This would have been fairly simple had it not been for the requirement to add new rows to the form using ajax. I know all about embedded forms and use them a lot - they really speed work up - but adding fields or forms to a form using ajax caused me a lot of head-scratching. But, I figured it out using the
material linked to below and some heavy doses of trial and error :)

This post is based on the extremely enlightening work of Nacho Martin - kudos and thanks to him for pointing me towards the light. The reason for me reinventing the wheel is that his post is more about the Symfony 1.2 way of doing things and this caused problems in 1.4 (Such as using a form as an array within the form or action for example...) So onwards and upwards...

The schema is very simple for this particular problem as we're using sfDoctrineGuardPlugin to handle the needs of the user model so we'll concentrate  on the schema of the invitation table for this example.

Our invitation table needs to be able to simply save a name and an email address. We can extend it later if we need to provide recognition tokens, tracking, follow-up emails etc etc. But for now, we just want simple :) so...

// /data/doctrine/schema.yml

Inivation:
  user_id:   { type: integer(4), notnull: true } 
  # note the integer(4) to make the definition equal the sf_guard_table
  email:     { type: string(255), notnull: true, unique: true }
  name:      { type: string(255), notnull: true }
  relations:
    User:
      class:   sfGuardUser
      local:   user_id
      foreign: id
      type: one
      foreignType: many


Nothing complicated there - we've got an id (auto-set by Symfony), a name and email (so we know who to address our email to) and a relationship to the sfGuardUser object defined so that the Inivitation has one User and a User can have lots of Invitations.

Ok, so this is where I started forking from Nacho's contribution. In my situation, I didn't want to create or even update my entire User object even though I did want to associate it with the Invitations created. So I created a new form called MasterInvitationForm

// lib/form/MasterInvitationForm.class.php

class MasterInvitationForm extends sfGuardUserForm
{
    public function configure()
    {
        $this->useFields(array());
        
        $this->widgetSchema->setNameFormat('user[%s]');
        
        $subFormCount = sfConfig::get('app_invitation_number',5);
        
        for($i = 0; $i < $subFormCount; $++)
        {
            $this->addInvitationForm($i);
        }
        
        $this->mergePostValidator( new InvitationValidatorSchema());
    }
}


There's a few things that we can draw from the previous code block. First off, our MasterInvitationForm is actually extending our standard sfGuardUser form. This is so we utilise the magic of Symfony later on when it comes to saving our relationships... read on.

Secondly, I've built this with reusability in mind and have set up our number of invitations that we'll display by calling an option and setting a default. If the option doesn't exist, we'll get the default - but this allows us to use this script again without changing the core code. If that's one thing I've learnt while building apps - it's reuse reuse reuse!

Penultimately, you'll see a call to addInvitationForm() within the for loop. This will be very handy indeed - you'll see why in a moment.

And last but definitely not least, we've merged a new custom validator into Post Validator schema. This will enable the custom schema to run over all the values within the POST submissions. Again - this will be very useful, but that will come even later.

So let's have a look at our addInvitationForm() method next.

// lib/form/MasterInvitationForm.class.php

class MasterInvitationForm extends sfGuardUserForm
{
    //...
    
    public function addInvitationForm($key)
    {
        if($this->offsetExists('invitations')
        {
            $invitationsForm = $this->getEmbeddedForm('invitations');
        }
        else
        {
            $invitationsForm = new sfForm();
            $invitationsForm->widgetSchema->setNameFormat('user[invitations][%s]');
        }
        
        $invitation = new Invitation();
        $invitation->setInvitingUser($this->getObject());
        $form = new InvitationForm($invitation);
        
        $invitationsForm->embedForm($key, $form);
        
        $this->embedForm('invitations', $invitationsForm);
    }
}


The addInvitaionForm accepts one parameter which is used as the name of the form within the widget, validator and error schemas - once we've set that all up.. but let's see what's going on.

We basically check for the existence of the form named invitations within the embedded forms collection and if doesn't exist, we'll create a new form and set up it's naming schema so that we can work on all the values as one later on.

We next create a new Invitation object and assign it the form's object (set as the current user in the action - we'll see that later). Then we create a new InvitationForm (the standard doctrine one) using the Invitation obect we just created.

Embed this InvitationForm object into the form named invitations using the  and lastly, re-embed (or embed for the first time if we've just created it) the invitations form into the MasterInvitationForm object.

We also add a custom configuration to our InvitationForm object. We only want to use the name & email fields within the form and we also want to have our labels within the input not externally (NB: The way I'm about to do this is NOT the best way, but it was quick and dirty for this particular project...)

// lib/form/doctrine/InvitationForm.php

class InvitationForm extends BaseInvitationForm
{
    public function configure()
    {
        $this->useFields(array('name','email'));
        
        $this->setDefaults(array(
            'name'=>'Name',
            'email'=>'Email'
        ));
    }
}


As I said, not the best way to prepare the setup - but it worked for this situation.

OK - so now we've talked about the setup and preparing for no labels on screen - let's have a look at the view elements quickly.

I created a new module called invite and added an actions class with a new action. This would be the first view our user would see.

// apps/frontend/modules/invite/actions/actions.class.php

class inviteActions extends sfActions
{
    public function executeNew(sfWebRequest $request)
    {
        if(!$this->getUser()->isAuthenticated()) $this->redirect('@homepage');
        
        $this->form = new MasterInvitationForm($this->getUser()->getGuardUser());
        
        if($request->isMethod(sfRequest::POST))
        {
            $this->processForm();
        }
    }
    
    protected function processForm($request);
    {
        $params = $request->getParameter('user);
        
        $this->form->bind($params);
        
        if($this->form->isValid)
        {
            $invitation = $this->form->save();
            $this->redirect('@thankyou'); // you can define this yourself - or change it as you need to of course...
        }
    }
}


The view for this action is as follows:
// apps/frontend/modules/templates/newSuccess.php

<?php use_stylesheets_for_form($form) ?>
<?php use_javascripts_for_form($form) ?>
<?php $errors = $form->getErrorSchema()->getErrors(); ?>

<form action="<?php echo url_for('invite/new'); ?>" method="post">
    <div id="invitation_rows">
        <?php foreach($form->getEmbeddedForm('invitations') as $key=>$subForm): ?>
            <?php if($subForm->isHidden()) continue; ?>
            <?php include_partial('invitation_row',array(
                'form'=>$subForm,
                'key'=>$key,
                'errors'=>$errors
            ))?>
        <?php endforeach; ?>
    </div>
    
    <div class="form_submit">
        <input type="submit" name="submit" value="submit" />
        <a id="add_invitation">Add another row</a>
    </div>
</form>

<script type="text/javascript">
    function addInvitation(num)
    {
        var r = $.ajax({
            type: 'GET',
            url: '<?php echo (url_for('invite/ajaxAddInvitationRow'); ?>'+'?key='=$('#invitation_rows div[id^=invitation_]').length,
            async: false
        }).responseText;
        
        return r;
    }
    
    $().ready(function(){
        $('a#add_invitation').show().click(function(){
            $('div#invitation_rows').append(addInvitation($('input[name=user[invtations]]').length));
        });
    });
</script>


There's a partial in there as well so that we can use it with the ajax function we'll see in a minute. This is defined as
// apps/frontend/modules/invite/templates/_invitation_row.php

<?php if(isset($errors['invitations'])) $errors=$errors['invitations']; ?>
<div class="form_row" id="invitation_<?php echo $key; ?>">
<?php foreach($form as $field): ?>
    <?php if($field->isHidden()) continue; ?>
    <?php echo $field->render(); ?>
<?php endforeach; ?>
<?php if(isset($errors[$key]): ?>
    <span class="form_errors" id="invitation_error_<?php echo $key ?>">
        <?php echo $errors[$key]->getMessage(); ?>
    </span>
<?php endif; ?>
</div>


Quite a lot going on - and the sharp eyed will have noticed that we've added a call in the main template to an as-yet non-existent method in the inviteActions class.

Before we get to that though, I'll walk through what's going on

I won't bore you with drawing the forms manually - it's pretty standard, if basic, stuff but then we add a link to the form submit div and hide it in the stylesheet (so users without javascript won't get bothered by the offer of adding rows).

Under the form in the template, we output our javascript. There's probably a better way to do this, and when I find it, I'll update this :). But in the meantime, I did it like this so that I could write the url easily from PHP.

It's basic jQuery ajax so I'm not going to walk through the details of that either, suffice to say that we're throwing the count of the rows back to the new action that I mentioned earlier. Which we'll have a look at now.

// apps/frontend/modules/invite/actions/actions.class.php

class inviteActions extends sfActions
{
    // ...
    
    public function executeAjaxAddInvitationRow(sfWebRequest $request)
    {
        $this->forward404Unless($request->isXmlHttpRequest());
        
        $key = intval($request->getParameter('key'));
        
        $form = new MasterInvitationForm($this->getUser()->getGuardUser());
        
        $form->addInvitationForm($key);
        
        return $this->renderPartial('invitation_row', array(
            'form'=>$form->getEmbeddedForm('invitations')->offsetGet($key),
            'key'=>$key));
        ));
    }
}


So what we're doing here is basically:
 1) getting out of here if it's not an ajax request
 2) grabbing the key
 3) creating a new MasterInvitationForm using the current user
 4) calling the form's addInvitationForm() method we created earlier
 5) returning that invitation_row, passing the newly created form referenced by $key to it for rendering.

The result is a new row appears in the form when you click the link. Great.

But...

We haven't touched on validation yet. If you remember we added a new InvitationValidatorSchema object to the MasterInvitationForm way back at the beginning when we created that class.

It's about time we had a look at that now to get ever closer to having everything working.

// lib/validator/InvitationValidatorSchema.class.php

class InvitationValidatorSchema extends sfValidatorSchema
{
    public function configure($options=array(),$messages=array())
    {
        $this->addMessage('invalid', 'Please enter a name and valid email');
        $this->addMessage('required', 'Please enter a name and valid email');
    }
    
    public function doClean($values)
    {
        $errorSchema = new sfValidatorErrorSchema($this);
        foreach($values['invitations'] as $key => $invitation)
        {
            if((strtolower($invitation['name']) != 'name' && trim($invitation['name'])!='') || (strtolower($invitation['email']) != 'email' && trim($invitation['email'])!=''))
            {
                $namVal = new sfValidatorAnd(array(
                    new sfValidatorString(array(
                        'required'=>false
                    ), array(
                        'required'=>'Please enter a name and valid email'
                    )),
                    new sfValidatorBlacklist(array(
                        'required'=>false,
                        'forbidden_values'=>array('name'),
                        'case_sensitive'=>false
                    ), array(
                        'forbidden'=>'%value% is not allowed'
                    ))
                );
                
                $emailVal = new sfValidatorAnd(array(
                    new sfValidatorEmail(array(
                        'required'=>false
                    ), array(
                        'invalid'=>'%value% is not a valid email address',
                        'required'=>'Please enter a valid email address'
                    )),
                    new sfValidatorBlacklist(array(
                        'required'=>false,
                        'forbidden_values'=>array('email'),
                        'case_sensitive'=>false
                    ),array(
                        'forbidden'=>'%value% is not allowed'
                    ))
                );
                
                $errorSchemaLocal = new sfValidatorErrorSchema($this);
                try{
                    $invitation['name'] = $namVal->clean($invitation['name']);
                }
                catch(Exception $e)
                {
                    $errorSchemaLocal->addError($e, (string)$key);
                }
                
                try{
                    $invitation['email'] = $emailVal->clean($invitation['email']);
                }
                catch(Exception $e)
                {
                    $errorSchemaLocal->addError($e, (string)$key);
                }
                
                if(count($errorSchemaLocal))
                {
                    $errorSchema->addError($errorSchemaLocal, 'invitations');
                }
            }
            
            $values['invitations'][$key] = $invitation;
        }
        
        if(count($errorSchema)
        {
            throw new sfValidatorErrorSchema($this, $errorSchema);
        }
        
        return $values
    }
}


So this looks massive, but actually all we're doing is iterating over the anticipated values from the post array - in this case the invitations array - and building some new validators to perform the cleaning on the incoming values.

The basic method of the errorschema can be found on the Symfony site - I'll post a link here when I have time to go get it.

But that's it -except. What happens when we add a row and submit it? The fields may all be validated, but we're getting csrf errors as there are too many fields for the form.

The way to combat this is to override the bind method of our MasterInvitationClass

// lib/form/MasterInvitationForm.class.php

class MasterInvitationForm extends sfGuardUserForm
{

    // ...

    public function bind($taintedValues=null, $taintedFiles=null)
    {
        foreach($taintedValues['invitations'] as $key => $value)
        {
            if(!isset($this->widgetSchema['invitations']['key'])
            {
                $this->addInvitationsForm($key);
            }
            if(isset($value['name']) && strtolower($value['name']) == 'name')
            {
                $taintedValues['invitations'][$key]['name'] = '';
            }
            if(isset($value['email']) && strtolower($value['email']) == 'email')
            {
                $taintedValues['invitations'][$key]['email'] = '';
            }
            if($taintedValues['invitations'][$key]['name']!='' || $taintedValues['invitations'][$key]['email']!='')
            {
                $this->getEmbeddedForm('invitations')->bind($taintedValues['inviations']);
            }
            else
            {
                unset($this->embeddedForms['invitations'][$key]);
            }
        }
        
        parent::bind($taintedValues, $taintedFiles);
    }
}


If by If then...

We iterate over the invitations array in the taintedValues variable. In each case, if the key doesn't exist in the widgetSchema then we add it using the method we created much earlier.

Next we check to see if we still have labels set in the values. (To be fair, we probably ought to move that over to the bind function of the Invitations form but it's late and I want to finish this before bed :)) If we do, then we set them to an empty string so we can test for the empty strings

Lastly, if either of the values are not empty, we bind them to the invitations form - otherwise, we remove them both. This way, we don't get any extra fields errors or extra, empty records that we don't want.

There are ways to improve this without a doubt. I've left out a lot of stuff concerning basic data management and I've skirted round the proper way to do things here in the interest of speed.

WHEN I get time to come back and make some changes and improve this, I shall. Unitl then, night night - it's bed-time.