Example intermediate collation file and Sphinx charset_table file

December 2, 2010

Here’s an example of a human-editable collation table. This is a manually edited version of the file that the first script generated for MySQL’s utf8_general_ci. The second script uses it to generate a Spninx charset_table:

!	0021
#	0023
%	0025
',’	0027,2019
0	0030
1	0031
2	0032
3	0033
4	0034
5	0035
6	0036
7	0037
8	0038
9	0039
@	0040
A,a,À,Á,Â,Ã,Ä,Å,à,á,â,ã,ä,å,Ā,ā,Ă,ă,Ą,ą	0041,0061,00c0,00c1,00c2,00c3,00c4,00c5,00e0,00e1,00e2,00e3,00e4,00e5,0100,0101,0102,0103,0104,0105
B,b	0042,0062
C,c,Ç,ç,Ć,ć,Ĉ,ĉ,Ċ,ċ,Č,č	0043,0063,00c7,00e7,0106,0107,0108,0109,010a,010b,010c,010d
D,d,Ď,ď	0044,0064,010e,010f
E,e,È,É,Ê,Ë,è,é,ê,ë,Ē,ē,Ĕ,ĕ,Ė,ė,Ę,ę,Ě,ě	0045,0065,00c8,00c9,00ca,00cb,00e8,00e9,00ea,00eb,0112,0113,0114,0115,0116,0117,0118,0119,011a,011b
F,f	0046,0066
G,g,Ĝ,ĝ,Ğ,ğ,Ġ,ġ,Ģ,ģ	0047,0067,011c,011d,011e,011f,0120,0121,0122,0123
H,h,Ĥ,ĥ	0048,0068,0124,0125
I,i,Ì,Í,Î,Ï,ì,í,î,ï,Ĩ,ĩ,Ī,ī,Ĭ,ĭ,Į,į,İ,ı	0049,0069,00cc,00cd,00ce,00cf,00ec,00ed,00ee,00ef,0128,0129,012a,012b,012c,012d,012e,012f,0130,0131
J,j,Ĵ,ĵ	004a,006a,0134,0135
K,k,Ķ,ķ	004b,006b,0136,0137
L,l,Ĺ,ĺ,Ļ,ļ,Ľ,ľ	004c,006c,0139,013a,013b,013c,013d,013e
M,m	004d,006d
N,n,Ñ,ñ,Ń,ń,Ņ,ņ,Ň,ň	004e,006e,00d1,00f1,0143,0144,0145,0146,0147,0148
O,o,Ò,Ó,Ô,Õ,Ö,ò,ó,ô,õ,ö,Ō,ō,Ŏ,ŏ,Ő,ő	004f,006f,00d2,00d3,00d4,00d5,00d6,00f2,00f3,00f4,00f5,00f6,014c,014d,014e,014f,0150,0151
P,p	0050,0070
Q,q	0051,0071
R,r,Ŕ,ŕ,Ŗ,ŗ,Ř,ř	0052,0072,0154,0155,0156,0157,0158,0159
S,s,Ś,ś,Ŝ,ŝ,Ş,ş,Š,š,ſ	0053,0073,015a,015b,015c,015d,015e,015f,0160,0161,017f
T,t,Ţ,ţ,Ť,ť	0054,0074,0162,0163,0164,0165
U,u,Ù,Ú,Û,Ü,ù,ú,û,ü,Ũ,ũ,Ū,ū,Ŭ,ŭ,Ů,ů,Ű,ű,Ų,ų	0055,0075,00d9,00da,00db,00dc,00f9,00fa,00fb,00fc,0168,0169,016a,016b,016c,016d,016e,016f,0170,0171,0172,0173
V,v	0056,0076
W,w,Ŵ,ŵ	0057,0077,0174,0175
X,x	0058,0078
Y,y,Ý,ý,ÿ,Ŷ,ŷ,Ÿ	0059,0079,00dd,00fd,00ff,0176,0177,0178
Z,z,Ź,ź,Ż,ż,Ž,ž	005a,007a,0179,017a,017b,017c,017d,017e
~	007e
Æ,æ	00c6,00e6
Ð,ð	00d0,00f0
Ø,ø	00d8,00f8
Þ,þ	00de,00fe
ß	00df
Đ,đ	0110,0111
Ħ,ħ	0126,0127
IJ,ij	0132,0133
ĸ	0138
Ŀ,ŀ	013f,0140
Ł,ł	0141,0142
ʼn	0149
Ŋ,ŋ	014a,014b
Œ,œ	0152,0153
Ŧ,ŧ	0166,0167
µ	00b5

And here’s a corresponding Sphinx charset_table from the second script:

charset_table = U+021, U+023, U+025, U+027, U+030..U+039, U+040..U+05a, U+07e, U+0b5, U+0c6, \
	U+0d0, U+0d8, U+0de, U+0df, U+110, U+126, U+132, U+138, U+13f, U+141, U+149, U+14a, \
	U+166, U+2019->U+027, U+061->U+041, U+0c0->U+041, U+0c1->U+041, U+0c2->U+041, \
	U+0c3->U+041, U+0c4->U+041, U+0c5->U+041, U+0e0->U+041, U+0e1->U+041, U+0e2->U+041, \
	U+0e3->U+041, U+0e4->U+041, U+0e5->U+041, U+100->U+041, U+101->U+041, U+102->U+041, \
	U+103->U+041, U+104->U+041, U+105->U+041, U+062->U+042, U+063->U+043, U+0c7->U+043, \
	U+0e7->U+043, U+106->U+043, U+107->U+043, U+108->U+043, U+109->U+043, U+10a->U+043, \
	U+10b->U+043, U+10c->U+043, U+10d->U+043, U+064->U+044, U+10e->U+044, U+10f->U+044, \
	U+065->U+045, U+0c8->U+045, U+0c9->U+045, U+0ca->U+045, U+0cb->U+045, U+0e8->U+045, \
	U+0e9->U+045, U+0ea->U+045, U+0eb->U+045, U+112->U+045, U+113->U+045, U+114->U+045, \
	U+115->U+045, U+116->U+045, U+117->U+045, U+118->U+045, U+119->U+045, U+11a->U+045, \
	U+11b->U+045, U+066->U+046, U+067->U+047, U+11c->U+047, U+11d->U+047, U+11e->U+047, \
	U+11f->U+047, U+120->U+047, U+121->U+047, U+122->U+047, U+123->U+047, U+068->U+048, \
	U+124->U+048, U+125->U+048, U+069->U+049, U+0cc->U+049, U+0cd->U+049, U+0ce->U+049, \
	U+0cf->U+049, U+0ec->U+049, U+0ed->U+049, U+0ee->U+049, U+0ef->U+049, U+128->U+049, \
	U+129->U+049, U+12a->U+049, U+12b->U+049, U+12c->U+049, U+12d->U+049, U+12e->U+049, \
	U+12f->U+049, U+130->U+049, U+131->U+049, U+06a->U+04a, U+134->U+04a, U+135->U+04a, \
	U+06b->U+04b, U+136->U+04b, U+137->U+04b, U+06c->U+04c, U+139->U+04c, U+13a->U+04c, \
	U+13b->U+04c, U+13c->U+04c, U+13d->U+04c, U+13e->U+04c, U+06d->U+04d, U+06e->U+04e, \
	U+0d1->U+04e, U+0f1->U+04e, U+143->U+04e, U+144->U+04e, U+145->U+04e, U+146->U+04e, \
	U+147->U+04e, U+148->U+04e, U+06f->U+04f, U+0d2->U+04f, U+0d3->U+04f, U+0d4->U+04f, \
	U+0d5->U+04f, U+0d6->U+04f, U+0f2->U+04f, U+0f3->U+04f, U+0f4->U+04f, U+0f5->U+04f, \
	U+0f6->U+04f, U+14c->U+04f, U+14d->U+04f, U+14e->U+04f, U+14f->U+04f, U+150->U+04f, \
	U+151->U+04f, U+070->U+050, U+071->U+051, U+072->U+052, U+154->U+052, U+155->U+052, \
	U+156->U+052, U+157->U+052, U+158->U+052, U+159->U+052, U+073->U+053, U+15a->U+053, \
	U+15b->U+053, U+15c->U+053, U+15d->U+053, U+15e->U+053, U+15f->U+053, U+160->U+053, \
	U+161->U+053, U+17f->U+053, U+074->U+054, U+162->U+054, U+163->U+054, U+164->U+054, \
	U+165->U+054, U+075->U+055, U+0d9->U+055, U+0da->U+055, U+0db->U+055, U+0dc->U+055, \
	U+0f9->U+055, U+0fa->U+055, U+0fb->U+055, U+0fc->U+055, U+168->U+055, U+169->U+055, \
	U+16a->U+055, U+16b->U+055, U+16c->U+055, U+16d->U+055, U+16e->U+055, U+16f->U+055, \
	U+170->U+055, U+171->U+055, U+172->U+055, U+173->U+055, U+076->U+056, U+077->U+057, \
	U+174->U+057, U+175->U+057, U+078->U+058, U+079->U+059, U+0dd->U+059, U+0fd->U+059, \
	U+0ff->U+059, U+176->U+059, U+177->U+059, U+178->U+059, U+07a->U+05a, U+179->U+05a, \
	U+17a->U+05a, U+17b->U+05a, U+17c->U+05a, U+17d->U+05a, U+17e->U+05a, U+0e6->U+0c6, \
	U+0f0->U+0d0, U+0f8->U+0d8, U+0fe->U+0de, U+111->U+110, U+127->U+126, U+133->U+132, \
	U+140->U+13f, U+142->U+141, U+14b->U+14a, U+153->U+152, U+167->U+166
Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: