Histogram based Pagination: A new way to paginate results
by soumadri on April 24, 2012
I am sick of the way Google and Facebook pushes results to us, with a fancy name of ‘Relevance’. Relevance ordering is good as long as it assists the user in finding results important to the user, not pushing results what the application thinks important to me. As rightly explained by Eli Periser in his TED Talk that this would lead to a filter bubble, and abolish the concept of ‘Open Web’. The purpose of web is to give us accumulated knowledge of the world, not only from my locality or similar parameters. The purpose of a search engine should be to give us all matched results, relevant only by the keywords, and the ranking of the page against that, not based on where I am located (unless its a location based service), or what I generally look for in past 6 months. These types of relevance filters are always should be a visible ‘opt-in’ in the search result page, where the user has control over what to see.
I was thinking on this for quite some time to solve this problem in an unobtrusive way, and I think I have found it. The concept is rethinking the pagination bar. 1, 2, 3 type pagination bar is a thing of past. The solution I propose would replace that and would also address the filter bubble problem from a user experience standpoint.
This solution is simple; replace the pagination bar with a histogram of relevant results w.r.t pages. This histogram would have a page slider with it. So, search result would appear unfiltered to the user and the histogram would indicate that in which page the ‘search engine thinks’ more relevant results are lied. Important thing to note here is the application is just assisting the user, not forcing the user to see what it thinks important, hence getting the user out of filter bubble.
Following is a design of the histogram based pagination bar (click/tap to enlarge):
The histogram in the back indicates the relevance per page and the slider is used to jump to specific page.
Following is a screen design of a search result page with histogram page slider:
Message to Google and Facebook: “Please understand that, machines are machines and do not force it’s understanding on us”
Drawing on iPad
by soumadri on December 27, 2011
Trying out this amazing app, Penultimate on iPad. This is one of the sketches.
Hurry Om!!!
by soumadri on December 12, 2011
Few days back, I was reviewing one of my colleague’s code and noticed that he used many unnecesary log messages (it was plain sysouts, not even using a logger). I told him to remove those statements immidiately. Not to my surprise, he told me “Ye to bas mein check karne ke lie dala hu. Baad mein hata dunga.” (I just added these to check something. I will remove these later.). Although it looks like a mere lazy nature of many developers, but actually, it is because we are too much in a hurry to finish the features, without noticing these tiny stuffs which we leave here and there. When it is the time to deliver the code, we forget to remove those stuffs, which most of the times becomes a teething trouble and reduces the quality of the software in general.
While coming back from office, I realized that, this phenomenon is not only applicable to the developers, but to almost all the people. Look out on the streets; everyone is in a hurry. Noone has the time to wait at traffic lights. Sometimes I think, we should have invented the “shortest path algorithm”, not Dijkstra. We are like ninjas in finding shortest possible path to reach the destination (albeit that creates a big chaos on road). Be it cycles, bikes or cars, we are smart enough to find a shortcut and leave everyone in a deadlock on the streets. When there is a traffic jam, we honk like crazy, as if honking would magically make all traffic disappear and make way for you. Do we have blessings of Moses, to cross the sea of traffic? Have you noticed house painters? Most of them are also are plagued with this. They paint in a hurry and splatters paint everywhere. If you ask them to clean it immidiately, you’ll get a similar answer that, they’ll do it later. Unfortunately that “Later” never comes.
In real life, the result of mindless hurrying results in visible chaos, but in programs this makes thing even worse. It results in invisible and untraceable shitty codes. It makes tracking bugs even more difficult.
Anyways, this phenonmenon is there to stay with us, but atleast if we would have invented the “shortest path” algorithm, I would have felt happier.
Creative Programming : Giving life to code
by soumadri on November 20, 2011
“Software development is a kissing cousin of engineering (if not an engineering discipline itself), and blends creativity with math and science. That’s why I find that a lot of advice to creative types is also applicable to software developers.” - Joey deVilla
Creativity is something which we generally do not relate to programming/software development. Typically we consider software development to be highly algorithmic in nature, where you can define what you want and how you can implement it. But essentially, its not the only thing which defines a software. I think software development is one such field where invention happens every now and then. I am not talking about new technologies being released; I am talking about the programs that we write everyday. Everyday we write something unique, for our product or project. But why are we so reluctant to recognize these creations? Partly because, these things are often taken for granted by most of the customers. Some ( or many) cases it is even worse, as the managers also do the same. This phenomenon is so dominant that many non-developers think anything is possible in software and it can be developed in “no time”. Especially this is applicable to IT services, where big fat non-IT customers pay big chunk of money to expect a miracle to happen, as if overnight they will become a market leader or reduce their production cost or whatever reason they are developing the system for. Although not all customers are like this, but most of them are. In this mess; most of the time; the creativity remains unnoticed. Even the developers themselves don’t realize that they actually created something to be proud of.
Creativity is highly relative. Don’t get deceived by thinking that, it is just about implementing any software. All developers do that. The difference lies in creating original idea and problem solving.
Original idea: Sometimes, a simple idea makes a big difference. This is not only applicable to features of a software, but also the way softwares are implemented. Many times this has something to do with mixing different technologies/approaches. Its the uniqueness of how you have solved a problem.
Problem solving: Many times it turns out that the conventional tools are not sufficient. Many developers just beat around the bush, hoping to get some solution. But creative ones, can differentiate between what’s possible and what’s not. When it is not possible, they create something on their own. I always believe that developers who go by their intuition get things right in this department, than sheer intellectuals. Definitely, being intellectual helps, but there are cases when you also need intuition of how things should work. This helps to create radically new solutions to many problems, which otherwise wasn’t possible. For some reasons, we Indians give extremely high importance to intellect, talent and analytical power. But that isn’t enough when it comes to creativity and innovation [this is a big topic of discussion]. Walter Isaacson wrote in NYTimes:
“China and India are likely to produce many rigorous analytical thinkers and knowledgeable technologists. But smart and educated people don’t always spawn innovation. America’s advantage, if it continues to have one, will be that it can produce people who are also more creative and imaginative, those who know how to stand at the intersection of the humanities and the sciences. That is the formula for true innovation, as Steve Jobs’s career showed.”
The expectations (and possibilities) from a software system has changed dramatically in last 10 years. Initially it was thrilling to create a “xxx management system”. As the technologies made advancement in terms of libraries and frameworks, it is neither thrilling to customers nor it is challenging enough. So how can we make a difference then? This should be addressed inside the software. Its the way we are implementing, the technologies/tools we are using and creating to solve the problem in hand. Customers will barely notice this, but when things “just works”, it makes that difference. The software doesn’t necessarily need a mind blowing feature. Whatever it has, it should just work the way it is expected and be as simple to the user as possible. This seems very easy but it is arguably the hardest thing in software development. We constantly need to look for new ways of solving problem, come up with innovative techniques to solve a problem. Only then we can stay ahead.
While creativity cannot be measured, there are some factors which drives a developer to be more creative:
1. Be passionate about what you are developing
2. Try to make the software beautiful from inside. Pretty, smart and less code always matters.
3. Do not re-invent the wheel, but don’t hesitate to invent whenever necessary.
4. Try to see the bigger picture.
5. Don’t just code the module you are responsible for, and shut your eyes when you see others code.
6. Break and disassemble others’ code.
7. Accept honest criticism of your code.
8. Don’t be an evangelist, be a pragmatic technologist, even though “evangelist” sounds cool 
.
9. Do not forget to get your hands dirty with technology and code, no matter how many years of experience you have in your pocket.
10. “Stay hungry, Stay foolish”. (Shamelessly copied from Steve Jobs’ speech )
There are awful lot of people (atleast in IT services) who are neither creative, nor they fall in the category of developers, even though they are paid to be so. Leaving this lot aside, others are unaware of their own creation. There has been a lot of discussion around quality of the software made by Indian IT service providers. While its a broad area to touch upon, the most basic thing which we can do to improve the scenario, is creative programming. No matter what kind of project it is, there will always be a scope for innovation. While most of the IT crowd need to learn this the hard way, the rest of them need to realize that they are in middle of creativity and technology. Blending both in some way, is their duty, as programming without creativity is lifeless.
Socket.IO library on Windows
by soumadri on November 6, 2011
This weekend was full of excitement with my test drive of Node.JS. One of the key capabilities of Node is ability to handle persistent connections and Socket.IO (NodeJS module) makes it a breeze. But it seems the Socket.IO package needs some hacks to run with Node’s windows binary, as NPM (Node Package Manger) is not available on Windows. It basically throws exception in resolving the dependency on socket.io-client and uglify. We need to manually download these two libraries and edit the socket.io code a bit to point to these. Here is the fully working Socket.IO library for Node.JS on Windows.
Extract the gzip and copy the ‘socket.io’ directory to the same directory as ’node.exe’. The directory structure should be as follows:
. | - lib | | - socket.io | - node.exe
To use this in your Node script, ‘require’ it as follows:
var io = require(‘./lib/socket.io‘);
On the client side you just need to include the ‘socket.io.js’ as:
<script type="text/javascript" src="http://localhost:8337/socket.io/socket.io.js"></script>
localhost:8337 is the <host>:<port> where my Node http server script is listening to.
Happy Node-ing
University suggestor tool
by soumadri on May 23, 2011
This is an effort to collect data related to MS, which would help in creating a University suggestor tool for MS. Please go through the link http://bs.thesubconsciousblogger.in/condor/ and submit your MS profile data. Your information is highly confidential (with optional name and email addresses).
Appreciate your help in submitting the data, and helping the other aspiring MS students.
The Real Facebook
by soumadri on March 19, 2011
My earlier article was about Facebook being a place to attract millions of people in not so nice way. Having said that, as everything has a flipside, Facebook is nothing different. While on the negative side, more and more people are getting addicted to the platform, the positive side is that slowly it is becoming a de-facto standard for online identity management. Although people are not giving much attention to this, a little observation on various web based services reveals that facebook authentication is almost ubiquitous. Starting from newspaper companies to various user profile oriented internet services are using facebook as their SSO (Single Sign On) mechanism. This offloads the identity management tasks from the service provider and provides all benefits of SSO to the end user. Although Google Accounts also provides this feature, people feel that their facebook profile is more real than their google account, which really has no added value to the user.
Facebook started from US university campuses, and naturally from start it had originality of profiles. Facebook founders were really serious in putting effort to make sure that the created profiles are not fake, unlike Myspace and Friendster. They even ranked profiles without photos, lower in the profile search. This worked so well over the time that even girls don’t hesitate much to put up their real photo in Facebook profile while this isn’t the case with Orkut and Myspace.
In the start of millenium there were quite a lot of services to manage virtual stuffs, Flickr for photos, GMail for mails but there were really no place to manage users’ online identity. Facebook precisely targeted this area. Although facebook is the coolest way to stay connected with your friend, the real self of Facebook is user’s online identity. Closely looking at all features features, it reveals. that the profile capture many information which reveals the true self of a person. The news feed keeps track of your interactions with your close friends and serves the news accordingly. Over the time facebook is building an information graph of your identity. From your profile and social graph, one can easily tell what kind of person are you, with what interests and what kind of people you are close to. While theoretically in any social network site this is possible, facebook really succeeded in this respect. Originality of profiles is more in facebook than any other social networking site. I think this is the core reason why more people are signing up in facebook. In India, Orkut lost the game mainly because of the same reason. I personally shifted to Facebook because of this. When a user joins a social network they look for only two things
- Whether the present profiles are original with frequent activities
- Whether most of their friends are present in it or not
If these two things are present, people gets attracted. Everything else merely is a way to lure the user to the platform OR to make them stick to the platform. If you seriously compare Facebook features to Orkut when Facebook just started in India, they were almost same. Initially facebook was a closed system, but they discovered the real value in making it open. Now, users and internet services are connecting, using Facebook.
Who knows, the global intelligence forces will start collaborating with Facebook to reveal user information, compromising our privacy?
Update:
I was reading today’s newspaper and found US military actually started keeping eye on Facebook using a software they bought. Here is the link.
The Facebook Avatar
by soumadri on December 23, 2010
Facebook started as an amazing way to connect to your friends, and to be social as a netizen. Slowly it evolved into a platform, which became a monetizing platform for many services. We discovered the true value Facebook is giving us, in addition to social networking. So far so good. But lately, I have noticed a very familiar evolution is happening with Facebook, which we have seen in another popular media, Television. If you observe the pattern, the way it has evolved, from a great platform to reach out to people to today’s “crowd pulling” TV; which is nothing but a marketing tool.
Let’s dissect the topic a little. In the early days of television, it was a very good platform to reach out to people. Many companies used it as a medium and made money out of it. Slowly the rate of advertisement increased and the quality of the programs on TV started deteriorating. TV started losing it’s goodness. Now most (not all) TV channels are full with idiotic “crowd pulling” programs, which is absolutely “unhealthy” for most of us, although we don’t notice this. It started with TV soaps,which elongates the episodes just in a desperate attempt to hold the audience to the channel, even if there is no story left. Then the reality show boom started, where most of the things are scripted, or the topics are wisely chosen to have huge on-screen battle. On-screen disputes between the participants are huge “crowd puller”. It has no purpose, other than attracting audience in a mean way. If you ask yourself “why to watch these programs?”, you will end up in no answer; because it has nothing in it.
A very similar phenomenon is happening with Facebook as well. It started as a great media, people started using it in their everyday life. Then something bad started to happen, when Facebook provided the platform to the world to creates “apps” for it. Initially, apps were pretty relevant, then some idiotic apps like “Lifebox”, “Fortune Cookie” and “Who will kill you” started to appear in everyone’s profile. I checked the page of “Who will kill you” app, and I was blown away by the statistics. This app has a 3.4/5 rating, 285733 active monthly users, 17997 likes. How in world an app with such an idiotic idea of “picking a most interacted friend who may kill you and when”, is getting so much of hits. It is earning $70-$100 per day. Thats a huge money. Icing on the cake is, people’s profile started getting more hits. So, more and more people are adding this application to become popular. This is the truth, “pulling crowd by doing something totally weird, which has absolutely no meaning”.
One can get publicity by doing two kinds of things, either by doing something really meaningful, which benefits others OR do something really weird/bad, like “getting naked on street”, to get even more attention. But the question is, “how you want to make a mark on others mind”, you want to be a scar on face OR mark in their heart. I think, Facebook has revealed its true avatar now. From a “social network” to “social attractor”, the evolution is complete now. The sad part is, other than providing networking , it is nothing but a place to “attract” millions of people by “any means”, which is same what television stands today in our day-to-day life. Facebook could have been a better e-Place to live, but slowly it is getting infected by the real social “popularity ” virus.
TableX – A simple jQuery expandable grid
by soumadri on December 4, 2010
For last few months I was finding a jQuery based expandable grid. My requirement was just a simple straight forward grid with row expand option. After searching a lot I couldn’t find a single simple enough option. Most of them are with too many options, dependency on several other stuff. Few were pretty good but requires AJAX to feed in the data, where the grid control itself loads and manages the data objects and communication with server. I didn’t want that as it introduces another URL to be exposed.
So, I created this simple jQuery Plugin to do my stuff. It’s pretty predictable. Its deals with the user interface only. It does that by transforming a simple table into an expandable grid. So, all you have to do is populate the HTML table using any vanilla server scripting technique, and just load this plugin which will change its user experience to a grid with expandable rows. You can download the plugin from, http://code.google.com/p/tablex . The “Getting Started” guide is available at http://code.google.com/p/tablex/wiki/GettingStarted. This plugin is open source and is available under GPL3 MIT.
The database nirvana
by soumadri on November 19, 2010
"The Database Nirvana"
What comes to your mind first when you hear the word “Database”? Let me guess….Oracle (the big daddy), SQL Server (clever uncle), MySQL (buddy), DB2 (grandpa) etc. Right? For most people I met, these systems define the term ”Database”. Let me reiterate the definition of database, “A database is a collection of data for one or more multiple uses”. Extending it to DBMS definition (Database Management System), “a software system that facilitates the creation and maintenance and use of an electronic database”. Does it anywhere mentions that a DBMS is relational. No. No documentation says that. Because relational DBMS is just a type of DBMS. Then why people still cannot/do not want to go beyond the DOMS (DB2-Oracle-MySQL-SQL Server) ? This question haunted me for quite some time. Few days back, while recalling my study materials from undergrad, I realized that the culprit entered into our mind at a very early stage. I don’t remember any famous DBMS books explaining anything other than RDBMS. This particular thing started it all. The university subjects are labeled as “DBMS”, but the text books are all explaining RDBMS, making DBMS=RDBMS thought ubiquitous. We used to bang our heads to learn “Relational Algebra”, while it was just one way to think about data. Slowly it invaded our mind and we established the thought DBMS=RDBMS; and unfortunately the impact has been so intense that still many people find it difficult to accept anything other than RDBMS. Some of them are so biased to the thought that, whatever dataset is presented to them, they start creating the “Table Terror” (design-normalize cycle).
While I was in a discussion related to some XML information stuff, few of my colleagues commented that “we can achieve the same using an RDBMS”. This proves how much penetration RDBMS did in their mind that, every data model they see, they start thinking in relational term always. But the actual fact is, there are lot many things in the programming world, that could have been done in a different way or by using some other technology/software. What matters is, “does it make sense”. Does it make sense to load XML data into RDBMS by “XML shredding”? Never. Never, this would have served as an alternative when we didn’t have any means to store XML but not now!!
The truth is we are so badly stuck in the RDBMS world, that we could barely think of any other kind of system managing our data. I am not denying the importance of RDBMS, as it is very important for extremely structured information. But if someone tries to put emails and documents in RDBMS, I would say it will surely be a waste of an effort. I was reading Dave Kellog’s blog the other day about the “Information Continuum“. He rightly explains that, the bulk of the information in this age is not highly structured (emails, tweets, blog posts etc.). Managing this huge semi-structured information is a real challenge. What makes it more strenuous is the mindset of “solving every data model with relational model”. This is high time that we take a different approach while thinking about DBMS and set expectations of what we can expect from a DBMS. Not a single DBMS fits all. Every situation, every new data model need something what “make sense”, not some desperate attempt to store the information anyway. Regarding the undergrad subject called “DBMS”, I feel, it must broaden its horizon to put some light on different DBMS (non-relational DBs) which exist and doing exceedingly well than RDBMS in their respective usage scenario (read this MongoDB vs. SQL Server report). Non-relational databases are also important, and demand attention from everyone. Google wouldn’t have reached today’s state, had it used RDBMS to manage its data instead of GFS. It’s time for a ”Database Nirvana”.
