Fixing UTF-8 Encoding when using the PHP ORM Idiorm

I love Idiorm and Paris as a ORM for my PHP Web apps.
Recently i run in a problem that in PHPMYAdmin Japanese text was not displayed correctly.

The cause was obviously bad encoding.
After a long time, I managed to find the solution.

First make sure that the mysql db is set to UTF-8 by default:
Add the following in my.cnf below the mysqld header:

[mysqld]
character-set-server=utf8
collation-server=utf8_unicode_ci

The next step is to set the collation to uftf_unicode_ci by default.
There is a setting on the first page of phpmyadmin and in the operations tab of every table.
Make sure these are all set to the collation utf8_unicode_ci.
(utf8_unicode_ci is apparently better than others in sorting. It allows sorting correctly in many many languages, but is a bit slower than the other collations)

Now for the most important, you might have figured the other things out for yourselves:

PDO needs to have it's connection string like this:

$dbh = new PDO('mysql:host=localhost;dbname=test;charset=utf8'$user$pass);

The important part is the charset=utf8 in the connection string.

So this means you should set up idiorm in your config file like this, with the additional charset=utf8:

ORM::configure('mysql:host=localhost;dbname=databasename;charset=utf8');

Now wait! There's more

So now you can write to the db and the new entries in the table are good. The old entries are gibberish in the app now however.
Solution: We have to correct the UTF8 double encoding that happened before. 
Credits go to Fixing double-encoded UTF-8 data in MySQL

Here is how to fix it, in two simple steps, using the mysqldump and mysql commands:

mysqldump -h DB_HOST -u DB_USER -p DB_PASSWORD --opt --quote-names \
    --skip-set-charset --default-character-set=latin1 DB_NAME > DB_NAME-dump.sql

mysql -h DB_HOST -u DB_USER -p DB_PASSWORD \
    --default-character-set=utf8 DB_NAME < DB_NAME-dump.sql

Of course, you should first replace DB_HOSTDB_USERDB_PASSWORD and DB_NAME with values, corresponding to your database setup.

 

 

I hope that helped you.

 

Written by Andreas Wildi on Sunday December 15, 2013
Permalink -

« My visit to Takamori Dengaku Hozonkai Restaurant 高森田楽保存会 - SSL Configuration on Apache »