Monday, February 15, 2010

Delayed Awakening



QPid jar chaos. The underlying of JNDI implementation - is it jvm-wide ? Clearly having single data/object lookup is a convenient way for managing objects within jar conglomerates. Ideally - all objects on the same jvm would have access to the same lookup.However, the regular question s popup - locking / deadlock conditions / performance / .. JNDI is used by JDBC/JMS/EJB. With JMS we use JNDI to register JMS administered objects.

commons-collections framework / Bag interface & others. An essential toolchain entry. Maven neverending check from updates on timeouting apache repo. Wasted time. QPid test app finally working on a minimum jar-bag. Benchmarking qpid stability/performance with random queue / entry generation. The real danger are the unhandled exceptions. Stupid maven pom update. Dynamic nature of JMS env can also be a challenge for stability. Finally bored of apache repo stalls, adding -o flag to maven builds. Offline it is.

Back to QPid world. Figureout the addressing, user/conf etc. When connecting to qpid server - the following is needed : (user, pass, clientid, host, pass, message_queue, routing_key, ?).

Thinking about efficient way to organize all of the personal info sources : (quantnet, willmot forums, ny times, google reader stuff, irc chats, dzone) ; rss is a part of the solution - integrated to google reader, however - no solution for non-rss sources + google reader does not reflect the fudamental way I would like to search these sources, no "fairness" mode - at least not clearly visible, more clear UI is needed, a different way to represent sources visually) - need integrated bookmarks + events somehow. Reader home page is ok, but does not solve anything, though it is a model. Extension of graph-placing-like problems to general visual representation (via constraint minimization). Relating that to portfolio optimzation (abstractly) ?

QPid topologies : p2p (client creates named queue , published publishes to queue with key mapping to queue name, client consumes) one-to-many (bind/publish to 'fanout' exchange), pub-sub (consume streams matching a pattern ?), fast-reliable-messaging, transactional, transient (non-durable messages), durable (with header defining durability), federation (link brokers with qpid-route and then create static/dynamic routes between brokers, resulting in graph-like structure). Going with p2p/named queues for starters. Very cool is a perftest provided with the qpid. (for java use QpidBench). Interested in message-size-dependency of throughput/latency. Don't forget to setup persistence store / benchmark performance vs durability etc.

Idea - rss reader with ranking of the feeds. Interaction + learning => optimal information dashboard.

qpid / gues/gues is a valid default user/pass, which leaves us only with determining connection/routing stuff.

issues with lifetime/scope of jndi Context object. " A Context instance is not guaranteed to be synchronized against concurrent access by multiple threads. " - interesting. Issues with Context as global variable - quazi-solution by creating it locally.

install artifacts locally, awesome! :

mvn install:install-file -Dfile=lib/qpid-all.jar -DgroupId=org.apache.qpid -DartifactId=qpid-all -Dversion=0.5 -DgeneratePom=true -Dpackaging=jar

The jms/qpid p2p queue up&running. Some adhoc benchmarks - 1000 messages / 2129 ms (including println); 100k messages - out of memory (client side clobber) ; 10k messages - slowdown on receive ; Simple pub/sub does not solve the 'dequeue' problem - we need to delete entries explicitly.

Checking Sun's JMS documentation, trying to figure simple enqueue/dequeue with persistency process. Definitely in the PTP domain. Revisiting fundamentals - ConnectionFactory creating Connection using Destination-described destination, via MessageProducer/MessageConsumer, created from current Session. ConnectionFactory and Destination are found via JNDI.
PTP : (QueueConnectionFactory, QueueConnection, Queue, QueueSession, QueueSender, QueueReceiver + (TemporaryQueue), QueueBrowser)

QueueBrowser as a example of scalable access to distributed resources. size() is an overkill, iterator wins. Not really working in this case, though.

Late night flipside - voidbase work. Integrate online news sources and let it roll. The call for the night - working token frequency for arbitrary source.

The pulse of the world : http://twitter.com/statuses/public_timeline.rss