Open Query blog » database

Open Query training at Drupal DownUnder 2012

arjen — Wed, 07 Dec 2011 04:00:56 +0000

DrupalDownUnder 2012 will be held in Melbourne Australia 13-15 January. A great event, I’ve been to several of its predecessors. People there don’t care an awful lot for databases, but they do realise that sometimes it’s important to either learn more about it or talk to someone specialised in that field. And when discussing general infrastructure, resilience is quite relevant. Clients want a site to remain up, but keep costs low.

I will teach pre-conference training sessions on the Friday at DDU:

MySQL Query Design ($220 half-day)
MySQL Server Tuning ($220 half-day)

The material is made specific to Drupal developers and users. The query design skills, for instance, will help you with module development and designing Drupal Views. The two half-days can also be booked as a MySQL Training Pack for $395.

On Saturday afternoon in the main conference, I have a session Scaling out your Drupal and Database Infrastructure, Affordably covering the topics of resilience, ease of maintenance, and scaling.

I’m honoured to have been selected to do these sessions, I know there were plenty of submissions from excellent speakers. As with all Drupal conferences, attendees also vote on which submissions they would like to see.

After DDU I’m travelling on to Ballarat for LinuxConfAU 2012, where I’m not speaking in the main program this year, but will have sessions in the “High Availability and Storage” and “Business of Open Source” miniconfs. I’ll do another post on the former – the latter is not related to Open Query.

SQL Locking and Transactions – OSDC 2011 video

arjen — Thu, 24 Nov 2011 02:10:30 +0000

This recent session at OSDC 2011 Canberra is based on part of an Open Query training day, and (due to time constraints) without much of the usual interactivity, exercises and further MySQL specific detail. People liked it anyway, which is nice! The info as presented is not MySQL specific, it provides general insight in how databases implement concurrency and what trade-offs they make.

See http://2011.osdc.com.au/SQLL for the talk abstract.

LexisNexis open sources code for Hadoop alternative

arjen — Sat, 10 Sep 2011 10:36:20 +0000

http://gigaom.com/cloud/lexisnexis-open-sources-code-for-hadoop-alternative/

HPCC Systems has released the open source code of its data-processing software that it’s positioning as a better version of Hadoop. The code is available on Github, and it marks the commencement of HPCC Systems’ quest to build a community of developers underneath Hadoop’s expansive shadow.

A Stick Figure Guide to the Advanced Encryption Standard (AES)

arjen — Wed, 24 Aug 2011 00:24:42 +0000

http://www.moserware.com/2009/09/stick-figure-guide-to-advanced.html

Jeff Moser on software development

On Password Strength

arjen — Thu, 11 Aug 2011 00:38:17 +0000

XKCD (as usual) makes a very good point – this time about password strength, and I reckon it’s something app developers need to consider urgently. Geeks can debate the exact amount of entropy, but that’s not really the issue: insisting on mixed upper/lower and/or non-alpha and/or numerical components to a user password does not really improve security, and definitely makes life more difficult for users. So basically, the functions that do a “is this a strong password” should seriously reconsider their approach, particularly if they’re used to have the app decide whether to accept the password as “good enough” at all. Update: Jeff Preshing has written an xkcd password generator. Users probably should choose their own four words, but it’s a nice example and a similar method could be used by an app to give “password suggestions” that are still safe.

Challenge: identify this pattern in datadir

arjen — Mon, 02 Aug 2010 02:07:14 +0000

You take a look at someone’s MySQL (or MariaDB) data directory, and see

mysql foo bar -> foo

What’s the issue? Identify pattern.
What does it mean? Consequences.
Is there any way it can be safe and useful/usable? Describe.

Good luck!

The actual range and storage size of an INT

arjen — Mon, 29 Mar 2010 02:45:03 +0000

What’s the difference between INT(2) and INT(20) ? Not a lot. It’s about output formatting, which you’ll never encounter when talking with the server through an API (like you do from most app languages). The confusion stems from the fact that with CHAR(n) and VARCHAR(n), the (n) signifies the length or maximum length of that field. But for INT, the range and storage size is specified using different data types: TINYINT, SMALLINT, MEDIUMINT, INT (aka INTEGER), BIGINT. At Open Query we tend to pick on things like INT(2) when reviewing a client’s schema, because chances are that the developers/DBAs are working under a mistaken assumption and this could cause trouble somewhere – even if not in the exact spot where we pick on it. So it’s a case of pattern recognition. A very practical example of this comes from a client I worked with last week. I first spotted some harmless ones, we talked about it, and then we hit the jackpot: INT(22) or something, which in fact was storing a unix timestamp converted to int by the application, for the purpose of, wait for this, user’s birth date. There’s a number of things wrong with this, and the result is something that doesn’t work properly. Currently, the unix epoc/timestamp when stored in binary is a 32 bit unsigned integer, with a range from 1970-01-01 to somewhere in 2037. Note the unsigned qualifier, otherwise it already wraps around 2004.

if using signed, you’d currently only find out with users younger than 7 or so. You may be “lucky” to not have any, but kids are tech savvy so websites and systems in general may well have entries with kids younger than that.

using a timestamp for date-of-birth tells me that the developers are young well that’s relative, but in this: younger than 40. I was born in 1969, so I am very aware that it’s impossible to represent my birthdate in a unix timestamp! What dates do you test with? Your own, and people around you. ‘nuf said.
finally, INT(22) is still an INT, which for MySQL means 32 bits (4 bytes) and it happened to be signed also.

So, all in all, this wasn’t going to work. Exactly what would fail where would be highly app code (and date) dependent, but you can tell it needs a quick redesign anyway. I actually suggested checking the requirements whether having just a year would suffice for the intended use (can be stored in a YEAR(4) field), this reduces the amount of personal data stored and thus removes privacy concerns. Otherwise, a DATE field which can optionally be allowed to not have a day-of-month (i.e. only ask for year/month) as that again can be sufficient for the intended purpose.

Storage Miniconf Deadline Extended!

cafuego — Wed, 30 Sep 2009 05:09:20 +0000

The linux.conf.au organisers have given all miniconfs an additional few weeks to spruik for more proposal submissions, huzzah! So if you didn’t submit a proposal because you weren’t sure whether you’d be able to attend LCA2010, you now have until October 23 to convince your boss to send you and get your proposal in.

Storage miniconf at linux.conf.au 2010

cafuego — Mon, 24 Aug 2009 02:53:16 +0000

Since you were going to linux.conf.au 2010 in Wellington, NZ anyway in January of next year, you should submit a proposal to speak at the data storage and retrieval miniconf. If you have something to say about storage hardware, file systems, raid, lvm, databases or anything else linux or open source and storage related, please submit early and submit often! The call for proposals is open until September 28.

From Daily WTF: Death by Delete

arjen — Sat, 27 Jun 2009 10:03:23 +0000

The Daily WTF collects excellent tales from the real world. These days, the dismal dramatic sagas are often (at least in part) about mistakes involving databases; no surprise there, they’re so prolific… Anyway, if you can learn from other people’s mistakes, that’s cheap and efficient education! I thought I’d share today’s edition with you: it’s called Death by Delete. Read and enjoy^H^H^H^H^Hlearn.