{ StevenHaddox.com }

During my day-job this week I came across a situation where we had nice functioning code very near to our launch of a Rails application. We had just completed migrating over some historical data from another database (have I mentioned how much I love rake in the past, if not I may have to do a blog post about that too). Anyway, the data all migrated over in a matter of minutes and after invoking a few more custom rake tasks that were simple to write we had every record in the exact same position as it was originally.

The cozy aura of accomplishment had settled upon our little team when we soon discovered that all was not well in Rails-land. You see, we are processing data at defined intervals (on a per-record basis by an attribute on the object). We have a cron job setup to invoke a rake task that checks and sees if each record needs to be updated or not (I love .minutes.ago and built-in timestamp fields btw). Then the rake task fires off the method call to update with the new data as often as needed. After the update is complete we parse the new data and create a JSON string that is stored in the database (for performance over parsing every time we want to display it). We also do a combination of all the past data sets we’ve received (one for 24 and 72 hours each) and store those in JSON format to the database as well. This worked out great on all our sample instances where our objects didn’t have hundreds of data sets being merged and combined into one long JSON string.

However, once we get some live data our application started blowing up rather quickly. We came across an ‘rbuff’ error (or something weird and very vague in its meaning) and spent a little bit of time realizing that the string we were trying to write to the database was more than 64KB. We had set our fields to ‘text’ and it was truncating our data (and hence blowing up all our JSON.parse methods to get our strings back into Ruby hashes). The fix seemed fairly easy, we just had to change our column types from ‘text’ to ‘longtext’.

If you do a Google search you’ll find several ways to have MySQL create a column in the table with a longtext type in Rails. However, none of them worked for me (we tried :text, :limit => huge_number as well as :text, :limit => 64K+1, and several other methods we found online). I couldn’t find any documentation to indicate that Rails had added in support for :longtext as an actual column type as it seems to be MySQL specific. My pair suggested (for the 10th time) that we just put :longtext in our migration file instead of :text and see if it worked. I’m not as much of an off-the-cuff kind of guy as he is and I was skeptical it would work as so many blogs said to do it the ways we had been trying. Well I finally gave in and said, “Okay, well try it your way, but I bet you’re wrong.” We edited the migration to:

change_column :table, :column_name, :longtext

rake db:migrate and all of the sudden our MySQL table was showing a column type of ‘longtext’ as we had wanted all along. Needless to say I was rather embarrassed that this wasn’t better documented and I lost the bet to my pair. My question now is, does anyone know if :longtext as a column type is supported on other database types or is tied directly to MySQL with Rails? I can’t seem to find any documentation for this as a column type anywhere so I’m not sure if it is new to Rails 2.3.2 or if it just has been overlooked in the docs. As the searches I performed hadn’t brought anything back of any use for migrating my column to longtext I figured I should try and make sure I at least get a blog up about it to help anyone else who may be in a similar situation.

Share It: