MySQL Jokeri

Pitanje za milion dolara u nekom popularnom kvizu. Imate samo 15 sekundi za odgovor. Srecni ste jer ste izvukli pitanje iz vase omiljene oblasti 🙂

Kako obrisati sve korisnike ciji nick pocinje sa donjom crtom (‘_’).

I vas odgovor je naravno:

delete from users where username like '_%';

Vec mastate o raskalasnom zivotu daleko od kompjutera, mozda i neka jahtica pride (i sve sto ide u to lol). Glas voditelja vraca vas u surovu realnost. “Zao mi je, ovo nije tacan odgovor”. U neverici gledate vas query i milione koji su otisli u nepovrat.

Verovali ili ne, slicnu nevericu iskusio sam na svojoj kozi pre nekoliko veceri, tacnije 8. aprila. U tabeli sa nekoliko miliona generisanih domena uocen je bug (mala greska u regularnom izrazu) i nekako su generisani i domeni koji pocinju sa znakom “_”. Underscore (donja crta) naravno nije dozvoljen kod domena, bug je ispravljen ali treba obrisati i te nevalidne domene iz baze. Nista lakse, jedan brzi query i sve ce ubrzo biti pocisceno. Otvaram mysql klijent i bez mnogo razmisljanja kucam:

mysql> delete from result_domains where domain like '_%';
Query OK, 9035782 rows affected (9 min 57.35 sec)

WTF!?? Query je obrisao sve domene iz tabele. Ali kaaakooo???

I onda se setim. Donja crta – ‘_’ je poput ‘%’, takodje joker karakter koji za razliku od ‘%’ (koji menja ‘nula ili vise’ karaktera), ‘_’ menja tacno jedan karakter. Ne secam se kada sam ga poslednji put koristio (ako sam ga uopste koristio), ali znam da mi je on dosao glave.

Cimam admina na ICQ … treba mi backup, server taj i taj, tabela ta i ta, poslednji koji imamo … ASAP! Posle 30-tak sekundi admin se javlja “ok, poslednji koji imamo je od 8 marta. Gde da ti stavim?”. FUCK!!!

FUCK!!! FUCK!!! FUCK!!! FUUUUCK!!!

Ustajem od kompjutera i besno setam kroz sobu. Na postoji nacin da objasnim klijentu da sam jednim jedinim kverijem sjebao sate i sate mukotrpnog rada, podatke vredne verovatno hiljade dolara. Da imam utoku verovatno bih pao u iskusenje da pucam sebi u glavu. Mozda pre toga da sredim i admina? Kako god, sta je tu je, ne preostaje nista drugo nego da napisem email i objasnim svom klijentu sta se desilo.

Sedam ponovo za comp, ali umesto da otvorim thunderbird, cimam ponovo admina.

– Ja: “Jebote … kako se desilo da je poslednji backup star mesec dana???”
– Admin: “Kako to mislis? Ovo je backup od sinoc!?”
– Ja: “Rekao si 8 mart???”
– Admin: “Ups. Sorry, 8. April. Sad gledam, kreiran je pre samo par sati. Moze?”

I tako, my ass has been saved. Ali moglo je i biti drugacije. Mnogo drugacije.

Pouka price:

1) Budite ekstremno oprezni kada kucate nesto unutar mysql klijenta (ili phpmyadmina) koji barata sa live podacima
2) Ako morate da brisete/update-ujete nesto, uvek uradite prvo count nad istim podacima i istom where klauzom kako bi ste se uverili da je to bas ono sto ste zeleli.

Npr:

mysql> select count(*) from result_domains where domain like '_%';
+----------+
| count(*) |
+----------+
|  9035782 |
+----------+
1 row in set (5.27 sec)

(ups nesto ne valja, ovo ce obrisati sve domene koje imamo)

mysql> select count(*) from result_domains where domain like '\_%';
+----------+
| count(*) |
+----------+
|        0 |
+----------+
1 row in set (4.18 sec)

Aha, sada je sve ok 🙂

3) Uvek pravite redovan backup. Ako to za vas rade admini postarajte se da rade svoj posao kako treba. Cak i tada nije losa ideja da s vremena na vreme napravite sopstveni backup.

P.S. Problem sa pocetka price (matchovanje stringa koji pocinje sa ‘_’) resava se jednostavnim escapovanjem specijalnog karaktera. Dakle:

select foo from footable where somefield like '\_%';

ili ako ste ljubitelj regularnih izraza:

select foo from footable where somefield regexp '^_';

PHP 4 End of Life Announcement

From php.net

[13-Jul-2007]

Today it is exactly three years ago since PHP 5 has been released. In those three years it has seen many improvements over PHP 4. PHP 5 is fast, stable & production-ready and as PHP 6 is on the way, PHP 4 will be discontinued.

The PHP development team hereby announces that support for PHP 4 will continue until the end of this year only. After 2007-12-31 there will be no more releases of PHP 4.4. We will continue to make critical security fixes available on a case-by-case basis until 2008-08-08. Please use the rest of this year to make your application suitable to run on PHP 5.

For documentation on migration for PHP 4 to PHP 5, we would like to point you to our migration guide. There is additional information available in the PHP 5.0 to PHP 5.1 and PHP 5.1 to PHP 5.2 migration guides as well.

Amen to that. Finally this will move the rest of people to PHP5, so we can finally start using all those great PHP5 features without worrying that such code cannot be used on most of client servers. Looking forward to PHP6 now 🙂

MySQL – Backup of Big MyISAM Tables

If you ever dealt with backup of MySQL tables, you probably used mysqldump utility, which allows you to dump all data into some mytables.sql file, which you can backup somewhere, import to other MySQL server etc. I used this procedure for a thousands times so far, and generally it goes as follows:

export:

mysqldump -udinke -pmojpass -hhostname.of.server1 dbname table1 table2 tableN > dump_file.sql
mysql -udinke -pmojpass -hhostname.of.server2 dbname  < dump_file.sql

Sometimes there is a problem when you move data from new version of MySQL to old, and in that case you have to specify proper compability flag when running mysql dump (--compatible=name where name can be mysql323, mysql40, postgresql, oracle etc.).

Anyway, few days ago in order to move data from one MySQL to another, I had to dump some ... let say big mysql tables (about 10 tables, where each contained about 10 millions of records). After long lasted procedure (dump to file, gzip, scp to other server) I finally started import. However, after 3 hours (yes, three hours) instead of Linux prompt I got this:

[dinke@um-917 ~/public_html]$ mysql -udinke -p325ewfwt23rasf
keyword_discovery < es_miner_data.sql
ERROR 1582 (23000) at line 163833: Duplicate entry '1167548' for key
'PRIMARY'

WTF? I moved data from old server to new, which means import should run without compability problems. It could be indexes on old table were damaged, but instead to wait like 2 more hours in order to complete check & repair procedure, I've decided to create dump file, this time with ignore option, so all insert queries in dump file are "insert ignore", so in case of error like previous one, errors will be ignored. Not very smart, but those data are not really high sensitive, and I can afford to lose few records but can't afford to lose 10 hours for import!

So, dump, gzip, scp, import again ... which lasted long... loooooong .... so fucking long that after 3 hours after I started import I started to think about other solutions. And solution was dumb but effective. We simple moved all MySQL data files (*.MYI, *.MYD i *.frm) from one server to another, and then we run myisamchk in order to fix those tables because ... we didn't shutdown MySQL server during copy procedure which generally could cause some problems with data.

All in all, this procedure went very fast (the longest was actual copy from one host to another), and in less than half hour I got everything settled down.

At the end I came with conclusion that in case of really big tables, using tool like mysqldump is unaccepted as backup solution because it takes literally hours to complete. One of solution to that problem can be to copy MySQL data files like I did. That shouldn't be a problem, because tables are "platform safe" meaning, binary file created on one platform (ie *.MYI file on Linux) will work without problem when moved to other problem (ie. on Windows). The only one real problem is that if you copy data wihout MySQL shutdown, table files there are in some kind of "state of flux" which can be a problem, especialy if they are highly used in moment when you do copy. That's why you need to do myisamchk on it.