PHP based search engines

I'm researching search engine technologies these days for my website http://indianlawexpert.com which is basically a search engine for indian legal information. The search functionality at indianlawexpert.com would obviously be semantic, but the underlying technology is still a good old fashioned full text search engine. This post is to record some of the search engine implementations that I considered for this project -

MySQL Full Text Search

MySQL %LIKE% keyword

Zend Lucene Port (Part of Zend Framework)

Sphider

Writing my own custom search index from scratch

Non PHP offerings

Lucene

Swish-e

Try out Free Desktop search tools for Linux

http://www.freesoftwaremagazine.com/columns/desktop_search_tools_gnu_linux_tracker_recoll_strigi_deskbar

TODO:

Basically Beagle is old-hat. Try using Tracker/Strigi and see which suits you best.

The mosquite test gave me a MASSIVE headache

Take the mosquito test to find out how good your hearing is..

I could not hear 15, 16, and 17 KHz but was able to hear 18KHz.. weird.. Now I have a massive headache! :(

Plus that probably puts me in the 30-39 range according to the chart above :( :( I feel old..

Books to read

On Douglas Crockford's recommendation -

Read -

1. Javascript the good parts
2. Javascript the definitive guide

Javascript is the world's most misunderstood programming language

Douglas Crockford's website (he's a Yahoo guy) has an amazing collection of articles on one of my favourite languages - Javascript.

Read about how Javascript is the world's most misunderstood programming language, thanks in no little measure to the half hearted attempts at standardisation and due to a botched design process.

Read the article on public/private/privileged members in JS objects to understand how OO works in JS.

Read the article on Inheritence in JS to understand.. well.. inheritence in JS.

Read about how scheme is similar to JS in the article on The Little Javascripter.

Javascript Coding Conventions - Follow them religiously

Javascript / CSS optimisation software

JS Compression software -

http://www.bananascript.com/index.php
http://dean.edwards.name/packer/
http://developer.yahoo.com/yui/compressor/
http://shrinksafe.dojotoolkit.org/
http://www.crockford.com/javascript/jsmin.html


CSS Optimisation software -

http://www.codebeautifier.com/
http://www.cssoptimiser.com/
http://www.cleancss.com/

JS Based 2d Physics engine

Box2dJS is a great javascript based 2d physics engine. The home page includes some great demos.

It is based on the awesome Box 2d physics engine.

PS: The Box2d author recommends using Bullet if you need 3d physics. This is the same engine used in Blender.

Strongly typed json

TODO: Write a javascript based library for handling strongly typed json data.

More concretely - Write a function that can take a json literal and a javascript object and fill in the fields of the object with the json data.

Basically this buys you support for default values.

Update:

Another (maybe good thing) would be to somehow add object annotation to JSON literals without breaking javascript compatibility.

e.g. Currently if you want to encode a Point(x,y,z) in JSON it would be something like {x:10, y:20, z:30}. If you want to encode dimensions of a box Box(x,y,z) in JSON it would again be {x:10,y:20,z:30}. There should be some way to encode the fact the literal is of the Point type or of the Box type.

A simple but ugly solution -

Point = {type:"Point", data:{x:10,y:20,z:30}}
Box = {type:"Box", data:{x:10,y:20,z:30}}

But maybe a better format would be similar to -

Point = Point {x:10,y:20,z:30}
Box = Box {x:10,y:20,z:30}

Since this is invalid javascript maybe this is a good compromise -

Point = ["Point", {x:10,y:20,z:30}]

Box = ["Box", {x:10,y:20,z:30}]

Naaaah! Ugly still.. Keep thinking..

OK for the time being, use this syntax -

Point = {"Point":{x:10, y:20, z:30}}
Box = {"Box":{x:10, y:20, z:30}}

This basically conforms more closely to the realisation that an "Object" in is just a hash with a class label.

Now there needs to be a way to firmly distinguish

the data - {"Point":{x:10,y:20,z:30}}

from

the data - {x:10,y:20,z:30}
of the type - Point

This is possible only if it is mandatory to specify the class for all literals. Which is plain wrong.

Think think...

Getting real - a book

Getting real the book - read it free here - http://gettingreal.37signals.com/toc.php

TODO - Read it for real instead of just bookmarking it!

Javascript location

Yup! The picture above has nothing to do with this post's contents.

Everyone knows that Javascript has (read write) access to the URL of the current page as window.location.href but you have a bunch of other useful properties in the window.location object.

window.location.search stores everything including and after the '?' in the current url

window.location.hash stores everything including and after the '#' in the current url

window.location.hostname stores the hostname

window.location.host stores the hostname + the port number

window.location.pathname stores the pathname (without the host string)

Coolio.