Izumi Blog: Ralf - Ideas For Dev
Index: Home     | What Is Izumi | Misc Links   | Random Thoughts | Too Much To Read | The Rant Vault | Quotes
Dev:   Projects | Ideas For Dev | Concepts

Ideas For Dev lists several dev-related items

Not that I may do anything with them anyway

This page uses reverse-date ordering.
Site License And Disclaimer as well as contact information are available here.

$Id: IdeasForDev.izu,v 1.74 2006-12-26 21:21:23 ralf Exp $

«»  2008/06/14 «» Android  «»

Some random Android ideas:

Some game that is a cross between Bejeweled and Jawbreaker. Done: asqare.

Remote: a background process on the phone that listens for commands from an authenticated channel. Could be a good way to experiment with App Engine as a way to provide a web interface and store the commands. Commands could be to spawn a process on the device in a shell, get GPS location, get current image, get current screenshot. GPS location could be set to auto-record and then upload a track in google map or earth.

Proxy: run an HTTP proxy on the phone. Then use an ad-hoc wifi connection with a nearby machine and browse the web using the phone. Some SSH clients can also tunnel via an HTTP proxy. Another idea is to simply run an SSH daemon with port forwarding on the phone, and simply access that port from the local machine.

[permalink]


«»  2006/12/26 «» JavaScript Toolkits  «»

Worth exploring: Dojo toolkit. Dojo is used by the excellent YouOS.

I vaguely remember seeing other similar JavaScript toolkits. I should have a closer look. There is of course Scriptaculous, and Yahoo UI Library whereas the Google Web Toolkit has a different target (Java code that produces JavaScript.)

Of course when I see the code behind YouOS, I remember that the major showstopper for using JavaScript-based extensions is not the language itself (JavaScript being a more-than-honorable prototype-based language) but the absence of proper development environment and of course the complexity that cross-browser incompatibilities add (i.e. CSS/JS/specs bugs.)

In this context, having a more sophisticated and predictable environment for development makes more sense and it should easy the pain a lot. That is Java in the case of [GTW||http://code.google.com/webtoolkit/] or Ruby on Rails with RJS Templates.

Or of course SeasideAsync for Seaside if you're in the Squeak/SmallTalk camp.

[permalink]


«»  2006/11/09 «» User Agents  «»

That's what it looks like to be hit by the Google Search on a mobile:

> combined.log <
....google.com -  [09/Nov/2006:08:46:18 -0800] "GET /.izumi// HTTP/1.0" 200 1287 "" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.0; Google Wireless Transcoder;)"
> combined.log.lang <
....google.com  "en"    "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.0; Google Wireless Transcoder;)"

So this happens whenever Google Search is used on a mobile, typically a cell or a PDA. GWT actually reads the pages and format them to make them easier to read (as in mostly text, reduced images, etc.) In this case it makes most sense to take advantage of this by changing the current template of the site if possible in your CMS or content renderer. For example remove any background pictures and as much decoration as possible and use very standard HTML tags (H1, H2, etc.) to organize content. It wouldn't hurt to improve accessibility by making sure you don't have a CSS-based menu or a left/right side float menu -- just simple links either at the top or bottom of the page (ideally you want the menu links at the bottom so that the content is presented first and at the top you want a "Menu" anchor that jumps to be menu at the bottom.)

And here's when you make a direct connection with a Razor V3's internal browser:

> combined.log <
...-dmz.mycingular.net -  [09/Nov/2006:08:47:54 -0800] "GET /.izumi/ HTTP/1.1" 200 1287 "" "MOT-V3/0E.42.15R MIB/2.2.1 Profile/MIDP-2.0 Configuration/CLDC-1.0 UP.Link/6.3.0.0.0"
> combined.log.lang <
...-dmz.mycingular.net "en"    "MOT-V3/0E.42.15R MIB/2.2.1 Profile/MIDP-2.0 Configuration/CLDC-1.0 UP.Link/6.3.0.0.0"

(Update/Side note: as pointed later by JBQ, what we see here is not the user agent generated by the V3 browser but by the uplink gateway itself.)

Same simplification for formatting as previously apply. However here we have a lot more info to work with since we actually have the phone name although this means it's not possible to detect all devices (because the string format is not formalized and you don't want to limit yourself to specific vendors but instead stay generic.)

This problem is actually nicely described here: Tutorial about Detecting User Agent Types and Client Device Capabilities

The solution to have is to get the x-wap-profile header to determine the User Agent Profile (UAProf). Here's a good explanation: Using UAProf (User Agent Profile) to Detect User Agent Types and Device Capabilities

The only drawback for this method is that it's a bit more involved. For a given request, you need to find the profile header, if available. This is an URL so you need to actually go download the corresponding XML and parse the RDF. Clearly you don't want to do that for each request. One solution is to do it once for each new profile and cache the result somewhere (typically only what you're interested in finding out.)

Another way to see this is understanding what you are looking for. Unless you have specific rendering requirements (i.e. screen size), it doesn't really matter what the profile contains: you just want to know you're targetting a "small" device. So you could rely on the sole presence of the UAProf header and not bother actually fetching and parsing the profile itself.

A low-tech alternative is however not to bother with the UAProf in the first place and consider that everything that doesn't have a "standard" user agent is a small device with limited capabilities. This strategy can fail in a lot of interesting ways however since there's nothing "standard" about the user agent string -- note how the Google Wireless Transcoder above identified itsel as being Mozilla Compatible, MSIE 6 and running Windows NT 5.0 when neither of these are true -- it's just a trick to force servers to give it the content no matter what.

Yet another alternative is to look at the accept content header: it may indicate that WAP content is accepted. No desktop browser would reasonably send such a header.

All in all, the real question is: what do you want?.

If you serve a complex CSS-based or table-based layout, it can be interesting to detect an "embedded" device and offer a simplier template. And if this is the case, you are probably hurting people with disabilities as your site probably doesn't play well with reduced accessibility options. For example, did you try viewing your site in Lynx?

If however you provide some kind of web album serving a lot of pictures, detecting a mobile device can be interesting. In this case it's worth looking at the UAProf to get the screen size. The accepted content header would also tell you a lot about what kind of content is suitable -- can the device handle images or video in the first place? Which formats?

If you want to provide a dedicated experience for each device, the user agent header might be just good enough yet it's going to be a lot of customization work with lots of testing. For a good example, look at the excelent work done by [OSNews:http://www.osnews.com] to display summaries on WAP/PDA or adapt to a variety of browsers.

In all other cases, if you format your default layout properly, it could render "good enough" directly. However this doesn't have to be automatic, it could be handled by have server-side or cookie-based preferences (take GMail for example) -- it's just a better user experience overall if the site can downgrade itself intelligently yet it's probably a good idea to let users eventually choose their layout and save it in a cookie.

And of course there's always this, but that's a different story:

> combined.log <
....googlebot.com -  [09/Nov/2006:08:48:37 -0800] "GET /.izumi/Ralf/TheBlog.blog?s=20060323_nx HTTP/1.1" 200 5191 "" "Mediapartners-Google/2.1"

[permalink]


«»  2006/10/26 «» Prototype-based... not  «»

When I started tinkering with Hint, my original idea was to create a prototype-based language. Obviously I had played with Self several years ago (who hasn't?!), then with Squeak (again, who hasn't!??!), I had I had just toyed with the excellent Io language and I also had realized than JavaScript was actually a real language which turned to be prototype-based.

Yet I had not done anything serious with either language. I had just toyed with them and for this very reason I found them totally super mega cool.

Then one day I suddently said to myself "duh, prototype-based languages suck!"

How come?

OK first, that reaction was a bit exagerated. They don't suck. However to the newcomer there still a slight issue with these prototype-based languages: the inherent lack of control that you get when the language allows anyone to redefine anything at runtime. That can't be good. Big no no. Nothing reliable and accountable can be based on such a language.

Now fans of these languages are plain screaming and they can flame me, burn their computers and stop reading here. Others more educated will just read and see for themselves if I have a point or not.

So let's see. On one side we have "static" languages like C, where everything is written in stone in a text file and then feed to a compiler. Reflexion is a pure programmer's wish and there's only a slight dim dynamic light casted by the RTTI (when in-house policy doesn't plain forbid it.) And unless you manually fiddle with v-tables you won't be extending a class at runtime.

Then you have not-so dynamic languages like Java or Visual Basic and .Net where not everything is an object although a lot is and there is some reflexion yet somehow once a class is defined, that's it, it won't change.

And finally you have Self, Squeak/SmallTalk, JavaScript and Ruby were everything is really an object and you may have a class keyword or not but it's not really relevant because any class (if you have it) can be redefined at any moment and maybe you don't even have real classes, just objects that you clone. Python is actually somehow in this group even though most Python programmers may not do this kind of thing.

If you come from C/C++ or Java, things can get very weird then since basically any class that you can think of can actually change depending on the libraries being used. That is you use Ruby or Python and just because you imported some module, new members appear in base classes.

And then it can get even more weird in something like JavaScript where importing some source (say the famous prototype.js library) will actually add tons of new objects and members to existing ones.

At that point most people who are use to not-so-dynamic languages get really scared. They'll tell you it's not secure since you can't control what the underlying base system contains -- it's possible a rogue library gets imported (directly or indirectly) and redefines key behavior. And there's no way you can prepare your code for that. It may be malicious or voluntary. Your code may break.

Total fear! Hide in the closet!

Basically they will not allow this on their finely tuned production servers, the whole stuff is just some kind of academic joke as far as they are concerned.

OK yes this is somehow true. But in reality it's not really that bad. Think of how Squeak "applications" are actually VMs preloaded with the right set of objects. Think the same of a Ruby "environment" such as Rails. When using Rails, Ruby is stil present but the environment is no longer Ruby -- it is Rails, which syntax is so elegantly imbricated within Ruby that the limit is blury and hard to determine for the new-comer. My point is that yes Rails modifies key behaviors in core Ruby classes such as Integer -- however in fact it doesn't modify them as much as it extends them and the whole point is that from the programmer's perspective the environment isn't just "ruby" anymore, it's "ruby+rails" so it's all under control.

This flexibility logically scares a lot of people, especially if their life revolves around C/C++ or Java (because that's what they learnt in school and use at work and they don't bother trying to learn other languages.) It's seen as lacking control when in fact it is merely augmented flexibility.

In most projects, you work in a closed environment. Something you're familiar with after painfully learning the API bit per bit. So yes change can be surprising but it's not a "lack" of control. I'd argue that if you add new libraries to an existing project and you don't know what they do, then you have a core problem that no language is going to solve.

But then even in the C/C++ world one can find a similar analogy. Environments in C/C++ are called stdlibc -- or the lack of. Not all platforms offer a Posix-compatible standard lib C. This is especially true for embedded compilers or custom RTOS. In this case it's a bit more than adding a library or two -- it's changing the whole subsystem, sometimes even the compiler with its share of different bugs.

The bottom line is that you don't use a language as much as you use a complete environment with its own runtime libraries and if you want to target production system you need to be on top of things and know what's in your environment. In this context languages when any core functionality can be redefined on the fly is not to be feared since you should know which libraries you're using and what they do.

Now the fact that one may like the concept of having class definitions separate from object instances or not have the former at all is a whole different topic.

[permalink]


«»  2006/08/12 «» Motion  «»

Basic question: how to implement a webcam-based motion detector?

Grabbing frames from a webcam is no biggie. I've done this with Xeres already (I'm thinking C#-based experiment here, so Xeres' code is just fine if I get to fix the memory leak in the acquisition lib I was using.)

So OK I have a bunch of time-based video frames coming in. I.e. a video stream. Colors are irrelevant, we can work with Y luminance using Y=(3R+7G+1B)/11.

The typical strategy is to diff consecutive images. If there's a variation in luminance, there's some motion (we assume the webcam is stable and doesn't move.) I remember trying a while ago by just doing a sum of Y component then applying a diff threshold. This didn't work very well for several reasons.

The first reason is that webcams are notoriously noisy, so there was a noticeable amount of noise. So a first idea is to remove high-frequencies from the images to filter the noise. Luckily there's the excellent FFTW library, of which I have already made a .Net wrapper in RgbBench.

Next global illumination was an issue. I figure I can remove the DC of the image and only keep the AC component or simply remove low frequencies. So effectively we apply a band pass filter and regenerate the filtered image.

Now I still need to diff images and detect something changed. One way to do it would be to cut the image in block and operate on blocks rather than on the whole image. Diff each block individually (by subtracting pixels' components) and compute the average diff for all blocks. Only blocks which pixels have changed more than the average are interesting.

Now we need to cut corners wherever possible. A typical webcam image would be 320x240 or maybe 640x480. We can simply reduce this to 160x120 for processing (using a simple box average for merging pixels together) and manipulate 20x15=300 blocks of 8x8 pixels or 8x6=48 blocks of 20x20 pixels. This latter value may seem more interesting. The FFT and inverse FFT would be done on the 160x120 Y-component image and diffs done on the individual blocks.

To reduce power computation the idea is to run this f.ex. every 5-10 seconds only in "idle" mode. When a noticeable diff is found, run it say every second and actually capture the original images as well as mark the motion blocks.

Another good idea I saw earlier on motion under Linux is to be able to capture an image and use it as a gray scale mask (f.ex. 0=ignore... 255=keep) which can help reduce disturbances by removing areas of no interest or affecting weights to part of the image.

[permalink]


«»  2006/06/25 «» RNR  «»

RIG but not RIG... hey why not.

OK here's a very different idea: create a daemon that monitors image folders and generates actual static HTML files and previews on the fly. It should also delete obsolete ones and update only what changed. The generated HTML would contain the selected template style, pagination, etc. Then a web server can serve the static content.

The obvious advantage is that you're serving static content. That's also the obvious inconvenient. So that removes the ability of features like user-side preferences -- image size, sorting, etc.

You can still have user comments and user rating for example: to update, simply switch to a dynamic site with forms, and post the information to the daemon. The daemon can then update a meta-database and regenerate the appropriate page.

One more advantage is that it makes it trivial to snapshot the site -- simply backup all the generated pages.

How's that different from what I had in mind for RIG 2? No too much actually. In RIG 2 I already wanted a server that would handle content. Now the thing is that originally I wanted the server to generate the content on the fly following a request for that content and that the server would generate just what is needed by the web front-end to generate the page. Here instead the server generates the page directly in advance and it is served statically.

I'd like to point out that even though a page is static, we can still put some dynamic content in it using some simple AJAX. Typical examples would be a MOTD. There are many other dynamic effect we can achieve using JavaScript with no server side access, such as changing the CSS theme, the layout or displaying a summary of user comments and expand them on user demand. All these are "extra" features and auxiliary to the real content of the page.

[permalink]


«»  2006/05/29 «» Ruby and Rails  «»

You've probably read Why Ruby is an acceptable LISP. Unless you're not in that kind of things and thus can safely skip this post.

(Note that if you are in this kind of things, you may enjoy the following page steam: Amb => Continuation Explanation => Actors Model => Power Loops.)

So yes, programming in Ruby is pretty satisfactory. It's clear and concise. You can have blocks, lambdas, metaclasses and such. After Ruby, C++'s clumsy syntax looks like an S&M exercise, Python looks silly with his need to declare self everywhere and PHP... Well my "$" key stopped working a while ago.

Of course once the over-enthusiasm has fallen down like a poorly baked soufflé, I'm left wondering if things are as green on the other side as it has been mentioned.

Obviously nobody's going to advertise a language by pointing out its defaults. Unfortunately I wish they would. This would stop short many language flame wars. After all, how many times do people have to complain that Ruby is not that much of an improvement over Python? There are some closed spheres where this is true and the benefice of Ruby does not outweigh nor justify the effort of a fourth implementation of everything over and over again. As far as I am concerned, the benefice over Python is good enough to make it worth. There are those who complain about Ruby when they are ecstatic about Perl, and quite frankly here I totally fail to see the point -- Perl is ugly and this ugliness is counterproductive enough for me that I want nothing to do with Perl, period. And same than for PHP, Python and Perl, the argument that the next incompatible major version is going to address all their weaknesses is a non-argument -- if it was really compatible, it wouldn't solve anything and it will take time to get as widely deployed. So in the meantime people will be forced using the new version but programming towards the old one for backward compatibility.

If you want something new, just use something new. And Ruby is here and good enough (and even a bit better here and a bit lesser there.) It's not the fastest VM in the world but then you're not going to implement data crunching algorithms with this. It's not the panacea but hey, these are programming languages and they are never the panacea. There's always one more thing.

Now one of the biggest problem with Ruby is Rails. More than often a Ruby detractor will slightly shift his focus on why you don't need Rails when the argument started against Ruby. I find that infinitely amusing. As much as the Rails clones such as that big joke called Grails and the new Perl thingy.

The point is Rails does work. It has its share of annoyances, the major one being the pain that it is to deploy it under Debian and its fragile dependance with the current Ruby version. It has its share of conventions that you better follow, yet you can still work around them if you're really stubborn (but in this case, choosing Rails is definitely a bad choice.) It integrates nicely with the Ruby language (as long as you have the right version) and is not too complicated to deploy once you understand the choices and their associated issues. And it bring real RAD to the web. After all I didn't even like Ruby and Rails until I re-implemented a full featured PHP web site in a couple of days after failing to add a new feature to the crummy PHP code base for a week.

The bottom line? I offer none. I'll give a try at Ruby and Rails for home projects like I did with PHP, Java, C# and Python in the past, which I all rejected to various degrees for various reasons.

Eventually some day I'll find an ubiquitous language that offers a mix of Smalltalk and Scheme without a crazy syntax, with the expressiveness and down-to-the-metal aspects of C++ and a good sensible adequate framework that doesn't require a 20 MB download first.

[permalink]


«»  2006/04/25 «» Reinventing the wheel  «»

It's a well known problem in software engineering: developers have a tendency to reinvent the wheel. Why use someone's else framework when you can as easily roll your own? There's even an acronym for this: NIH (Not Invented Here.)

The causes for this syndrom are pretty much obvious: partly for ego (I can do better than this guy), or fame (I get all the credit), or control (it's my framework.) Most likely for all of the above in some or other degree.

Sometimes it makes sense -- that's called evolution. Maybe I can really do better than this guy. The plenitude of programming languages is a good example. That's why we have C++ vs. Java vs C#, Perl vs Ruby, etc. Would you rather still be programming in Pascal or even worse Cobol and Fortran? (On the other hand, what happened to APL?)

Most of the time it's useless and for one language that succeded to fame there are dozen who failed silently.

One non obvious problem with reinventing a language is the failure to realize the size of the task. It's easy to come up with some grammar rules, fire flex and bison and get something pronto. Yet depending on the target you want to reach you'll realize you need libraries, a runtime and most important a community and a commitment. Implementing the actual language is almost the most trivial part.

Same goes with frameworks. For example, imagine you stumble accross something like Rails. It's a pretty cool new framework, people love it and it shows an obvious successful track record. But then there's a major problem for Rails: it's not coded on my-favorite-language. Instead it's coded in someone's else favorite language and that's not good.

That's how you end up with Grails: the desire to reinvent the wheel despite the fact that the targetted language is not perfectly round and adequate, so it needs to be patched to look like... something else tham it was supposed to look like just for the sake of implementing the framework. Namely, Grails uses Java but it's not Java anymore... it's Groovy and Grails is just one many subprojects. The bottom-line is that it's something else and even if you know Java you can't code in Grails right away.

Grails is a very good example of reinventing the wheel for all the wrong reasons.

Even though, both Groovy and Grails have another big issue: the project is based on technical merit. The home page describes lots of technical reasons why it's cool. It fails to mention successful examples. Grails provides a couple tutorials which totally fail to introduce the concepts to newbies. Instead if you want to understand why Grails, you need to go to the Rails home page and read the introduction.

Documentation and support is the key, even in open source.

And finally Groovy fails for the same reason that Java: the project is already embryonic and crippled with tons of side-projects with stupid names. On the Ruby camp, you have Ruby and Rails and that's all. On the Groovy side you have Groovy, Groovy Markup, GPath, Groovlets, Groovy Sql, Groovy Beans, Grails, Gram, Groovy SOAP, etc.

[permalink]


«»  2005/11/12 «» New Structure for Project Files  «»

I'm going to try a different document structure for project files:

These projects are generally just ideas that I explore. As such a project may not be completely defined when I start it. Eventually the project will go dormant for a while (days, weeks or even years) and then suddenly I want to explore it again. Typically in this case I find something was wrong with the original goal and want to refocus the project differently. In this case I tend to create a new project, say FooBar II instead of FooBar I. But this is odd as the former project had never been completed and sometimes not even started (as in not implemented.)

In this case instead of thinking of the new project as a different project I should rather think of it as a new iteration. It's enough to use the same project file and label all the notes and intro and plan files as belonging to iteration n and then move on with iteration n+1

Sometimes I really want to have a totally different project. An example is RIG 2 versus RIG 1. RIG 1 is an existing PHP web app that is close to be final and matches or even already surpassed the initial goals. RIG 2 is a new experimental project with a totally different scope. In this case RIG 2 is not a derivation from RIG 1 and deserves to have its own project file. On the other hand the scope of RIG 2 has already been refined three times. These correspond to three different iterations of RIG 2 and as such should be part of the same document.

Consequently a number of project pages need to be refactored:

On potential derivation of this structure is to have one project page per milestone of a major project, for example I could have a main RIG 2 page that links to RigII-M0, RigII-M1, etc. This could be necessary if the project files becomes to big to the point that it's hard to read. In this case it would make sense for the main project file to contain the last milestone (for example RigII-M36) and then move the older milestones in their own file (RigII-M35, etc.) The main project file would link to the older milestones.

The last important point of the new document structure is the design & spec section. Originally these project pages were either plan files or design/specification pages. Lately they mostly centered around the implementation notes, which is a misnommer as many times these contain design comments.

Instead I need a clearly defined design & spec section for a given milestone. This must focus on structure and not be a time-ordered log. The section should at least contain part of this:

Each section should start with an overview & a rough sketch before any implementation happens and then get more detailed as the real implementation occurs. As the implementation goes, original thoughts may become obsolete and will need to be revised. The old paragraphs should be either removed or moved to an "obsoleted" section.

[permalink]


«»  2005/11/05 «» Live Database  «»

When I stopped coding on Xeres and Rivet last time, I had a similar issue: I wanted to completely dissociate the logic from the UI. Also I wanted to use some kind of callback or messaging mechanism to avoid tight coupling.

I remember now that I said something exactly like that for Hint: consider a program has a database of object. Viewing the program is just like querying a database.

So that's an idea. right there. It's not innovative in the sense that it probably has been done and there are surely things implemented this way out there. I don't care as long as it makes it easier for me to do what I want.

In the case of Xeres, there should be only a few live objects: media sources for audio and video, media consumers and UI consumers. I see two approaches here:

That's three approaches but the two last one are identical. Kinda.

In the case of Hint, we could say that each directory and file is a unique object in the database and the UI performs queries and display.

In any case, the mechanism can be abstracted as follows:

Consumer objects should be based on a monitoring/event model: they request a given live query on the database and are notified when objects are added or removed to this query. A granularity may be necessary.

Note for Xeres I started with a producer/consumer model with callback notifications. The model here is mostly similar except there's an intermediary in the middle. So technically that's one more layer and it's thus less efficient. Yet here I am interesting in having a generic model and remove as much direct coupling as I can.

Most notably I'd like to completly disable all links to and from the UI.

[permalink]


Blog Archives:
Most recent posts
2005/10/30 - 2005/06/07
2005/06/01 - 2005/04/03
2005/03/31 - 2004/12/18
2004/12/09 - 2004/07/05
2004/06/18 - 2004/03/28
2004/03/27 - 2004/01/18
2003/12/22 - 2003/11/02
2003/11/02 - 2003/09/01
[RSS]


Site License

Creative Commons License
This work is licensed by Raphaël Moll under a Creative Commons License.

Options
Color Theme: Gray  | Blue  | Black | Sand  | Khaki  | Egg  | None

Web ralf.alfray.com Powered by Google

Display Izumi & PHP Credits

Stats
1595 accesses, 1 access from 38.107.191.82
Visited 110 times by Google, last 2010/08/17 20:35
Visited 239 times by Yahoo!, last 2010/08/26 20:38
Visited 1 times by Alexa, last 2008/11/16 06:56
Visited 29 times by Teoma, last 2010/08/10 15:01
Visited 237 times by MSN, last 2010/09/01 19:47

< Generated in 1.38 seconds the 09/03/2010, 09:35 AM by Izumi 1.1.4 >