Abstracting Data in JavaScript
I have recently been considering developing a framework to radically simplify the development of user-interfaces. Warning: This post is long and nerdy. You probably won’t want to read it. There are no pictures.
The History
I’ve been working as lead UI developer for CourseTopia for about two years now. It has been by-far the largest and most involved project I have ever worked on. I am also planning on developing several simple web apps over the next year as portfolio work to be used in internship-seeking.
As such, it seems reasonable to come up with a solid planning process for creating web apps. Creating CourseTopia was a great learning experience, but going into it, I had no idea exactly what kind of work user interfaces can be.
We toyed around with implementing a system like this for CourseTopia a while ago, but ultimately decided to continue working on the site as we were. Someday, the site may be rewritten with this technique, but we are unsure.
I am going to describe the essential design behind this application creation framework. The actual code necessary to write the framework would be minimal. It’s really more of a way to think about user interface development. Please note that this has not been worked out in its entirety, so suggested modifications are welcomed.
The Problem
The main problem that I faced time and time again, is a lack of abstraction in my code. It seems to be a process of figuring out exactly what parts of the interface need to be modified when some action occurs. This is also a source of a great number of bugs in the code, which may or may not show themselves for a long time after the initial code has been written.
User interfaces, in general, are a way to display and interact with data. The basic principle behind this paradigm is abstracting away data dependencies so that the user interface is, at all times, a current reflection of the data state itself.
Traditionally, the way we handled this with CourseTopia was to say, I want to perform an action, from which I expect a certain response. When I perform this action, I will make some immediate changes to the page, and then when I get the response, I will update everything in the UI that could possibly be affected.
The problem with this, is that it is very difficult to conceptually organize all the dependencies of a certain change in data. This makes performing simple actions incredibly difficult, while at the same time a source of many bugs which are not immediately discoverable.
Introducing State
So to solve this problem, we introduce the idea of data state. It is no radical new programming concept. State is merely the current value of all pieces of information in existence in the system.
The main idea behind this paradigm is abstracting away the work of making the interface a real-time reflection of the state of the system.
For this we can create a class of objects, called dataObjects. dataObjects would be responsible for maintaining some type of data. The data could be anything from a true/false value, to a big chunk of text, to a list of items.
This data would also maintain which obects depended on it, so that when any change was made to the data, those objects could be updated.
Data inside the object could be accessed by some convenient method, for now .get(). They could also be modified by the .set(obj) method. It’s all fairly intuitive, but behind the scenes, a call to .set(obj) is making a call to all things that depend on this piece of data.
I have also considered creating some type of templating system for these dataObject, so that you can expect certain pieces of information to be included. On the other hand, this might not be necessary. Another possible consideration is the situation where one piece of data might depend on another. In that case, there might need to be some kind of an update() method…
The UI Elements
In general, user interfaces are composed of several high-level components. The components can very easily by hierarchical, a component being made of several other smaller components. For this reason, we introduce the concept of the uiComponent.
uiComponents are things that might be updated by a change in data. They are generated entirely on the fly, so they are intended to be used for things that a search-engine might not necessarily be interested in crawling.
I am currently thinking that the actual code used to create uiComponents should be the responsibility of the UI developers, according to some interface.
I think the only real necessary methods would be render() and update(). render() would be used to render the component initially, building DOM elements on the fly according to however the developer chooses. update(), on the other hand, would be use to make changes to the data.
It is reasonable not to force developers to declare an update() method, if the act of updating is not necessarily different than the initial render.
It is also necessary to somehow collect a list of dependencies, which dataObjects this object depends on. This list might need to be declared in the constructor, or perhaps could even be built dynamically as pieces of data are used. That would be ideal. Either way, the dataObjects that this component depends on should be notified somehow so that they can update when necessary. Perhaps this could be done by adding a parameter to the dataObject.get() method…
I have also considered the performance advantages of building the dependency list as not just a list of strings, but a list of string, function fields. The idea would be that some pieces of data which get regularly updated might have some more specific way of updating that don’t require a potentially expensive call to the update() method. On the other hand, however, this could also be accomplished by creating heirarchies better.
An alternative to the interface-type design would be to create anclass uiComponent that accepts render/update functions as arguments for its constructor, and then have some method to place them into the DOM somewhere. This might make dynamic updates a little easier.
I am unsure as to what should necessarily be done for certain components which may appear several times on a page. It might be reasonable to add a parameter, context, to the constructor method. I am unsure at this point. To clarify, this paradigm is still extremely early in development, more just ideas.
On Actions
If we could also abstract away the actions people can perform, it might make handling such things as queueing and error-handling a lot easier. For instance, we might create a uiAction interface that can handle whatever occurs.
Abstracting out the things that occur in every action: we make some changes right away, this could be a parameter called doNow. This may be all that occurs, if no server communication is necessary. However, other actions may require a server call, so a call field is necessary, along with certain post parameters. Once a response from the server is received, there may be certain things that need changed, so a success field could be in order. Also, in the event of some type of failure, network or otherwise, there might need to be some handleError(e) method that accepts an error object and decides what to do with it to notify the user.
These actions could be chained in such a way that every time a call is made to the server, it sends a list of all actions that need to occur. The server will then process the actions, and return the results all at once.
Another nice feature would be to add an optional undo function to handle the ability to undo the action, in a reasonable way.
Navigation
Another nice useful interface would be a navigation interface, to control navigation items that might be handled by the back/forward button. I have not looked into exactly how this will work, but navigation objects should only be required to have a navigate() method that simply performs whatever navigation is necessary, in a timelesss way, so that it can easily be called by a browser’s back button request.
Communication
This brings me to communication. There is to be some single file which the system makes calls to. Any call to the server could theoretically contain several requests for action, so this server-side file acts as an index, pointing to other files to process the requests. Once all requests have been handled, all that will be returned is a list of dataObject names, along with associated values, serialized in some reasonable way. Also, any request failures would also be returned so that the handleError method could respond appropritately.
To Summarize
class dataObject : used to store a piece of information, which can be accessed by the interface
- new dataObject(name, data) : construct a new data object
- name : the name of the data object, must be unique or else an exception can be thrown
- data : the piece of information being stored
- public get() : retrieve whatever data is in this object
- public set(data) : set the data stored in this object, update all dependencies
- data : the value to be stored
- private dependencies : a list of all items that need updated when this data changes
interface uiComponent : used to create user interface components
- private dependencies : list of strings representing dataObjects that this component depends on
- public render() : initially place this object somewhere in the DOM
- public update() : update the DOM to reflect current data, if different from render
interface action : used to perform some type of action in a standardized way
- public doNow(): perform whatever parts of the action can be done immediately
- public call : what call to be made
- public parameters : what parameters to be passed along with the call
- public success() : what to do if the call is made successfully; should not include updates to data as this will be handled automatically
- public handleError() : how to handle any errors that might come about
- public undo() : how to undo this action
interface navigation : used to navigate from one area to another, that would be reasonable to integrate with the back button
- public navigate() : used to access a particular part of the application
That about covers it. I will keep you posted as to my development of this interface development plan, whether I decide to write code for it or not.
- name : the name to be used by this piece of data, must be unique
- data : the information stored in the object
Hi Kevin,
Just like you, I have quite a bit of experience in UI design in Javascript now. The HTML DOM provides a very neat platform for producing nice UI interfaces, although it also has its disadvantages because of browser differences. But, as you surely have found out as well, it is doable to support the most commonly used browser by sensible and careful usage of the HTML DOM, even in standards compliance mode.
The idea that you have shown here is really a framework which is built on the most general concepts of state machines. It may be handy for a beginner as an educative example, but I feel that, as general and low-level as it is, it’s still not well applicable to very advanced user interfaces. For example the action interface; it presumes that it should be undoable. But sometimes, it might save you a whole lot of work to let the undo-work be done by simply cloning the document. I know, it’s inefficient but in many occasions a better idea than putting the responsibility in an action. Especially if you expect more actions to come in newer versions.. It’s a trade-off, but in your framework there is no room for this.
Also, putting the parameters in a single attribute doesn’t feel right. It degrades code quality; I’d say it’s better to pass them into a well defined action-specific method on the object on which the action takes place, and use design by contract and careful code documentation to increase code readability.
My personal experience is that such a framework is overkill. Every UI is different, and designing them is a black art. Every time you have to puzzle with the bits and pieces of knowledge that you have obtained previous projects.
What I think would be more interesting, is to wrap up the DOM into commonly-used datamodel elements, for instance a Table, Select or List class. These can contain methods which are likely to be used (createExtraColumn(), addOption(option), selectOption(option), sortList(), etc.) by GUIs, and maintain their own references to the DOM tree. This is a bit like having a UI wrapping around a DOM branch, but using this framework you could hide away some of the cumbersome tasks for elements of the UI which are general and not standardly supported by the DOM.
wait for it..
I approve.