Dariush Debian Diary

using T/TCP for FTP transfers turns FTP from ideal 'File Transfer Protocol', into ideal 'Files Transfer Protocol' (by reducing cost(bandwidth and time) of transfering multiple files (RFC 1379) Similiar effect half-closing tcp might have...(??)

logs are precious, rrd believes otherwise, that wouldn't be a problem if rrd wasn't so popular. Thus, one needs tool that sits on top of rrd/modifies rrd, and stop it from erasing data. That is the most economical course of action, since every data logging tool outthere uses rrd.

How to proceed?

1. cron-fired tool that backs rrd-data into separate file
2. patch to rrd, that stores removed data in alternate data stream (another file), with a little effort this might become an option to rrd, making historical data easily available to rrd-based tools

- install reverse-munin, it reports stats every ~15m (stats collected every ~30s)

1. Divide your flash-based block device into >3 parts, for example let's use 12 sections.
2. Raid5 (or Raid6 if you need more protection) them.
3. use resulting block device as base for your filesystems/swap.

This way you can run without wear-levelling layer (which AFAIK can't work in general case, because you would need Size/blocksize bytes/nibbles of additional wear-resistant storage to keep statistics of your wear), AND you gain additional protection, for very little price ( less then 10% of your storage ).

Conventional wisdom says you shouldn't raid on single device, but since flash devices are not mechanical and don't suffer noticable seek penalty you can do this with very little loss of performance. If you'd do this in hardware inside your card, there would be no performance penalty.

Another problem is that raid5/raid6 are famouse for their poor write speed, but, since you're already writing whole blocks-at-a-time when dealing with flash devices, you're already paying the prices (writing block-at-a-time is the best case scenario for raid5/6, there's no need to recalculate checksums then)

1. Effective protection for wear problems, plus additional protection for other block errors
2. Very small price storage-wise (<10%)
3. Very small performance penalty when you synchronise flash blocks with raid blocks (the penalty for writing less-then-block-at-a-time is already payed when dealing with flash, raid penalty is virtually non-existant then)
4. Ideal for embedding in hardware, but doable on software-side

Note: it doesn't make much sense to use raid5 instead of raid4 in this case. And raid4e for double protection. TODO: implement RaidF as dm layer?

As I'm slowly getting older, I encounter recurring problems , with perl and general practices being already decades old, one would think that there are well-tested and generic solutions.

One of those thingies is problem of generic hmm, message-scheduler or maybe switch.

You can see this in projects like alamin (sms gateway), varios MTAs, B2B solutions sending invoices, SNMP managent consoles etc..

One way of solving that is by using filesystem - aka spool directory.

Popular solution - shared space aka spool, multiple daemons handling messages - incoming, outgoing, processing...

Two mains choices - spool directory and sql table.

Requirements for spool:

• Ability to drop incoming message into it
• Ability to check emptiness of spool/list waiting msgs
• atomic operation for changing status of msgs ( processing, finished )

18:04 <@muppet> to compile perl extensions on windows 18:04 <@muppet> you need the same compiler that was used to build perl 18:04 <@muppet> with activestate's perl, that's msvc's cl.exe 18:04 <@muppet> there's a free non-optimizing version of msvc available from microsoft's website. 18:05 <@muppet> you'll need also to get the platform sdk, because perl.h wants windows.h (memory, threads, and process management) 18:05 < eyck> oh my, that's hours and hours of searching the net and downloading stuff, sounds like fun. 18:05 <@muppet> if you have that, and the libglade win32 binaries, and the development headers for all of the gtk+/glade stuff, compiling the Gtk2::GladeXML extension is really easy. 18:05 <@muppet> not that much, i found the links for whitewindow in about five minutes 18:06 <@muppet> i don't have a windows pc here to do it on for myself, though. 18:07 <@muppet> how to get the free msvc7 compiler, with examples of compiling a perl module: http://howtos.beaucox.com/win32-vc7-compiler.html 18:08 <@muppet> all-in-one installer for gtk+, glade, and libglade on win32: http://gladewin32.sourceforge.net/ 18:09 <@muppet> whitewindow's Gtk2-Perl binaries: http://lists.gnome.org/archives/gtk-perl-list/2004-June/msg00182.html

On uwoo server runs dropbear 0.41:

ghost eyck 11:20 ~/ >time ssh uwoo  echo

ssh uwoo echo  0.03s user 0.00s system 18% cpu 0.161 total
ghost eyck 11:20 ~/ >time ssh uwoo  echo

ssh uwoo echo  0.04s user 0.00s system 22% cpu 0.182 total

On uwoo server runs openssh 3.4:

ghost eyck 11:20 ~/ >time ssh uwoo  echo

ssh uwoo echo  0.10s user 0.02s system 15% cpu 0.770 total
ghost eyck 11:21 ~/ >time ssh uwoo  echo

ssh uwoo echo  0.12s user 0.01s system 21% cpu 0.597 total

glibc-based woody with dropbear 0.41:

ghost eyck 11:28 ~ >time ssh dev  echo

ssh dev echo  0.02s user 0.02s system 16% cpu 0.244 total
ghost eyck 11:39 ~ >time ssh dev  echo

ssh dev echo  0.04s user 0.00s system 25% cpu 0.160 total
ghost eyck 11:39 ~ >time ssh dev  echo

ssh dev echo  0.03s user 0.00s system 17% cpu 0.172 total
ghost eyck 11:39 ~ >time ssh dev  echo

ssh dev echo  0.02s user 0.01s system 17% cpu 0.172 total

glibc-based woody with openssh 3.4:

ghost eyck 11:39 ~ >time ssh dev  echo

ssh dev echo  0.13s user 0.00s system 33% cpu 0.387 total
ghost eyck 11:40 ~ >time ssh dev  echo

ssh dev echo  0.11s user 0.00s system 27% cpu 0.403 total
ghost eyck 11:40 ~ >time ssh dev  echo

ssh dev echo  0.09s user 0.00s system 23% cpu 0.382 total
ghost eyck 11:40 ~ >


Test: code
Hello world

Hi againg
e - anything

I need perl module for accessing TLS-enabled ftp servers, unfortunately Net::FTP can't do that, and it's object model makes it painfull to implement that feature

See more ...

I'm in quest for 'asynchronous bi-directional' communication protocol.

Starting point are:

• IRC Internet Relay Chat, traditional workhorse in the field.
• CIMD2 Nokia protocol for communicating with SMSC/telephones. Has few nice properties and quirks.
• Jabber famous protocol that solves the problem - how to spy on people's chats in times when most IM's go encrypted.

asynchronous bi-directional - the most important thing, and different from most of the "normal" protocols, is the fact that system must be able to sand AND receive message in any given moment. "Normal" protocols like FTP/SMTP/POP3 etc, work this way:

client -> request -> server
client <- response <- server

so basicaly, on very low level of protocol - client can't receive anything from server until it asks for it. This something important in perl world - we've got Net::* clients, and many of those use object infrastructure of Net::CMD... this means that you're on your own if you want to create client for some IM...