Main Page | Report this Page
Linux Forum Index  »  Linux Development - System  »  Character Drivers in User-Space...
Page 1 of 2    Goto page 1, 2  Next

Character Drivers in User-Space...

Author Message
Bob Smith...
Posted: Wed Oct 14, 2009 11:41 pm
Guest
Hi

I am working on a Linux based robot controller and had to solve
the problem of creating a lot of Linux device drivers quickly.
My solution was to create a couple of real, but thin, device drivers
and use them as the front-end to a user-space driver daemon that
did the real work. I have written a short article about this work
and posted it at http://www.linuxtoys.org/usd/UserSpaceDrivers.html

I would appreciate any feedback that might make the article easier
to read, but I would _especially_ appreciate feedback on the two
kernel modules.


thanks
Bob Smith
 
Grant Edwards...
Posted: Thu Oct 15, 2009 7:08 am
Guest
On 2009-10-15, Bob Smith <bsmith at (no spam) pohl.linuxtoys.org> wrote:
Quote:
Rainer Weikusat wrote:

You'll probably be decorated, at least in mind, by the
GPL-avoidance-acrobats from the gadget industry,

This is certainly not my intent. All of our Linux software
will be open-sourced, probably with a BSD type license. For
us at least it is not a matter of license but of productivity.
We have to do a lot of drivers and could not possibly get them
all into the kernel.

Even if you could get them into the kernel, maintaining Linux
drivers is a big PITA with kernel APIs constantly shifting out
from under you. I've been dreaming for years about moving some
of the linux driver code I work with into user-space. However,
doing so would require that the pty driver be enhanced to act
like a serial port, and that offers to do that have always been
ignored.

--
Grant Edwards grante Yow! Should I get locked
at in the PRINCICAL'S
visi.com OFFICE today -- or have
a VASECTOMY??
 
Jan Kandziora...
Posted: Thu Oct 15, 2009 7:29 am
Guest
Bob Smith schrieb:
Quote:
I have written a short article about this work
and posted it at http://www.linuxtoys.org/usd/UserSpaceDrivers.html

Well, why do you use kernel level drivers at all? For your application, a

user level dameon with a unix socket or pty interface would suffice...

Kind regards

Jan
 
Rainer Weikusat...
Posted: Thu Oct 15, 2009 7:48 am
Guest
Bob Smith <bsmith at (no spam) linuxtoys.org> writes:
Quote:
I am working on a Linux based robot controller and had to solve
the problem of creating a lot of Linux device drivers quickly.
My solution was to create a couple of real, but thin, device drivers
and use them as the front-end to a user-space driver daemon that
did the real work. I have written a short article about this work
and posted it at http://www.linuxtoys.org/usd/UserSpaceDrivers.html

I would appreciate any feedback that might make the article easier
to read, but I would _especially_ appreciate feedback on the two
kernel modules.

You'll probably be decorated, at least in mind, by the
GPL-avoidance-acrobats from the gadget industry, since you have solved
on of their important problems (how can we use Linux and still keep
our users completely under our thumbs) for free.
 
Grant Edwards...
Posted: Thu Oct 15, 2009 9:52 am
Guest
On 2009-10-15, Bob Smith <bsmith at (no spam) pohl.linuxtoys.org> wrote:
Quote:
Jan Kandziora <jjj at (no spam) gmx.de> wrote:
But I don't think character devices are really "user friendly", except if
the user should use tools normally tied to such character devices, like
e.g. use minicom with it or dd a firmware file to it.
OK, but this might come down to personal preference. I like the
idea of telling a stepper motor to step 2000 steps with the command:
echo "2000" > /dev/dp/step_count

To do the equivalent with, for example, unix sockets would
probably require an API to the sockets, so a .so library to
hide the details of the socket.

I don't really understand what "API to the sockets" would be.
All the languages I know of already have socket APIs.

In any case, I'd probaly use IF_INET sockets instead of Unix
sockets. Then you can use netcat. If you're controlling stuff
with a shell script, then sending data through the "lo"
interface to the daemon is certainly going to be more than fast
enough. If you're using IF_INET sockets, then you also get
network transparancy for free. Your application doesn't even
need to know whether the daemon is on the same computer or is
on the other side of the planet:

echo "step 2000" | nc -q0 $dpHost $dpCmdPort

That assumes all commands go to the same port number. If you
allocate different ports for different "commands" (each port
would correspond to one of the char devices in your
implementation) you would do

echo "2000" | nc -q0 $dpHost $dpStepCountPort

The above uses TCP, but you can do UDP instead by adding "-u"
to the "nc" command's options. [I would probably only do UDP
for local networking.]

No kernel modules required, and you can write the daemon
entirely in Python or any other high level language.

Writing something like netcat for unix-domain sockets would
also be pretty trivial.

--
Grant Edwards grante Yow! VICARIOUSLY experience
at some reason to LIVE!!
visi.com
 
André Gillibert...
Posted: Thu Oct 15, 2009 9:58 am
Guest
Bob Smith <bsmith at (no spam) linuxtoys.org> wrote:
Quote:
Hi

I am working on a Linux based robot controller and had to solve
the problem of creating a lot of Linux device drivers quickly.
My solution was to create a couple of real, but thin, device drivers
and use them as the front-end to a user-space driver daemon that
did the real work. I have written a short article about this work
and posted it at http://www.linuxtoys.org/usd/UserSpaceDrivers.html

I would appreciate any feedback that might make the article easier
to read, but I would _especially_ appreciate feedback on the two
kernel modules.


Linux 2.6.31 supports CUSE (character device in user space).

--
André Gillibert
 
Bob Smith...
Posted: Thu Oct 15, 2009 10:46 am
Guest
Bob Smith <bsmith at (no spam) linuxtoys.org> wrote:
Quote:
My solution was to create a couple of real, but thin, device drivers
and use them as the front-end to a user-space driver daemon



Jan Kandziora wrote:
Quote:
Well, why do you use kernel level drivers at all? For your application, a
user level dameon with a unix socket or pty interface would suffice...
A valid point. We could have. There were several reasons why we

did not. Making the device look like a real device is, I think, a
little more user friendly. Also, neither sockets nor pty devices offer
a broadcast capability which is required, nor do they offer the flow
control that I wanted.



Rainer Weikusat wrote:
Quote:
You'll probably be decorated, at least in mind, by the
GPL-avoidance-acrobats from the gadget industry,
This is certainly not my intent. All of our Linux software will

be open-sourced, probably with a BSD type license. For us at least
it is not a matter of license but of productivity. We have to do a
lot of drivers and could not possibly get them all into the kernel.


André Gillibert wrote:
Quote:
Linux 2.6.31 supports CUSE (character device in user space).
Yeah, CUSE/FUSE was our "PLAN A" approach. But I could not get

select() to work at all, and good select() support is our number one
requirement. Also, CUSE and FUSE are fairly big and complex compared
to the fanout and proxy drivers. CUSE is a work in progress and by using
fanout/proxy I'm now ready to start into the daemon itself.


again, thanks
Bob Smith
 
Grant Edwards...
Posted: Thu Oct 15, 2009 11:11 am
Guest
On 2009-10-15, Bob Smith <bsmith at (no spam) pohl.linuxtoys.org> wrote:
Quote:
Grant Edwards <invalid at (no spam) invalid.invalid> wrote:

I don't really understand what "API to the sockets" would be.
All the languages I know of already have socket APIs.

Can you think of a common Linux daemon that offers only a
socket as the application interface? I can not.

No, but I see that as more of a design decision than some sort
of techincal resitriction.

Quote:
Most daemons that use sockets, such as DBus, do not allow raw
access to the socket and instead offer library that "hides"
the socket from the application. While not required, hiding
the socket behind a library is the most common Linux approach
and an approach I do not want to use.

That's probably because they use complex C "structs" for
transferring binary data. Using simple, easily generated/parsed
ASCII data formats eliminates much of the need for such an API
and provides something that's trivial to use from shell/Perl
scripts. [At the expense of the overhead required for
formatting/parsing.]

Quote:
Grant Edwards <invalid at (no spam) invalid.invalid> wrote:
In any case, I'd probaly use IF_INET sockets instead of Unix
sockets. Then you can use netcat.
echo "step 2000" | nc -q0 $dpHost $dpCmdPort
That assumes all commands go to the same port number.
Writing something like netcat for unix-domain sockets would
also be pretty trivial.

Grant, you are right and have given me pause to rethink my
whole approach.

[I don't know if I'd be all that quick to throw everything out
based on my off-the-cuff remarks.]

Quote:
There are, or will be, about 30 device driver to do and each
of these has several configuration parameters. This many
devices would make mapping from robot peripheral to TCP socket
number quite a problem. But unix sockets would not have this
problem.

Neither would specifying parameters in-band, but that's more
overhead as well.

Quote:
So, with your socket approach ...

ROBOT DRIVER FEATURE SET
- Unix domain socket with good bidirectional data flow
- Good bidirectional flow control
- Broadcast (not intrinsic, but easy to do in the daemon)

FWIW, UDP does support broadcast and multicast, though
attempting to do either over the "capital-I Internet" is
futile.

Quote:
- No root privledge to to set up
- No mucking with kernel modules

How big a hassle that last one turns out to be probably depends
on which kernel APIs you end up using. Some of them (e.g. the
tty layer) seem to be rather fluid.


--
Grant Edwards grante Yow! GOOD-NIGHT, everybody
at ... Now I have to go
visi.com administer FIRST-AID to my
pet LEISURE SUIT!!
 
Jan Kandziora...
Posted: Thu Oct 15, 2009 12:24 pm
Guest
Bob Smith schrieb:
Quote:

Jan Kandziora wrote:
Well, why do you use kernel level drivers at all? For your application, a
user level dameon with a unix socket or pty interface would suffice...
A valid point. We could have. There were several reasons why we
did not. Making the device look like a real device is, I think, a
little more user friendly.

Well, ok. You like to hide the fact the whole communication is done in

userspace.

Recently, I had to use socat to simulate a pty (hence the idea) to talk to a
distant IO server via TCP, as my favorite high-level language would only
offer timeouts on communication for character devices. I haven't wanted to
rework the whole application to be thread-safe, so that was a good
workaround.

But I don't think character devices are really "user friendly", except if
the user should use tools normally tied to such character devices, like
e.g. use minicom with it or dd a firmware file to it.


Quote:
Also, neither sockets nor pty devices offer
a broadcast capability which is required, nor do they offer the flow
control that I wanted.

Broadcast? What do you mean? Concurrent access?

Same for flow control. Please explain.

Kind regards

Jan
 
Bob Smith...
Posted: Thu Oct 15, 2009 12:58 pm
Guest
Jan Kandziora <jjj at (no spam) gmx.de> wrote:
Quote:
But I don't think character devices are really "user friendly", except if
the user should use tools normally tied to such character devices, like
e.g. use minicom with it or dd a firmware file to it.
OK, but this might come down to personal preference. I like the

idea of telling a stepper motor to step 2000 steps with the command:
echo "2000" > /dev/dp/step_count

To do the equivalent with, for example, unix sockets would probably
require an API to the sockets, so a .so library to hide the details
of the socket. But not just one .so file since to be well received
I'd need to write binding for my API for lots of languages -- PHP,
Perl, ... This does not sound "minimalist" at all.


Jan Kandziora <jjj at (no spam) gmx.de> wrote:
Quote:
Broadcast? What do you mean? Concurrent access?
Same for flow control. Please explain.
If you haven't already, would take a few minutes to read the

article at the URL in the original post.

The idea of device broadcast is not usually visible but is in the
/dev/input subsystem. An example in the article shows how to see
this. Robot sensors (my interest) also need a broadcast system.

I did not mention "concurrent access" but perhaps this refers to
the idea of broadcast? One writer and many concurrent readers?

Both named pipes and pty devices allow you to write up to 4K bytes
when just one end is open. I wanted to block all writes until
both ends were open. "Flow control" to me refers to select() in
a way. I need bidirectional transfer (ruling out named pipes)
and wanted flow control to flow in both directions up to the
select() on each side. One side being the driver daemon and the
other side being the high-level robot application. I hope this
makes sense.


thanks
Bob Smith
 
Bob Smith...
Posted: Thu Oct 15, 2009 2:53 pm
Guest
Grant Edwards <invalid at (no spam) invalid.invalid> wrote:
Quote:
I don't really understand what "API to the sockets" would be.
All the languages I know of already have socket APIs.
Can you think of a common Linux daemon that offers only a

socket as the application interface? I can not. Most daemons
that use sockets, such as DBus, do not allow raw access to the
socket and instead offer library that "hides" the socket from
the application. While not required, hiding the socket behind
a library is the most common Linux approach and an approach I
do not want to use.



Grant Edwards <invalid at (no spam) invalid.invalid> wrote:
Quote:
In any case, I'd probaly use IF_INET sockets instead of Unix
sockets. Then you can use netcat.
echo "step 2000" | nc -q0 $dpHost $dpCmdPort
That assumes all commands go to the same port number.
Writing something like netcat for unix-domain sockets would
also be pretty trivial.

Grant, you are right and have given me pause to rethink my
whole approach. There are, or will be, about 30 device driver
to do and each of these has several configuration parameters.
This many devices would make mapping from robot peripheral to
TCP socket number quite a problem. But unix sockets would not
have this problem. So, with your socke approach ...

ROBOT DRIVER FEATURE SET
- Unix domain socket with good bidirectional data flow
- Good bidirectional flow control
- Broadcast (not intrinsic, but easy to do in the daemon)
- No root privledge to to set up
- No mucking with kernel modules


Thanks.
Bob Smith
 
Bob Smith...
Posted: Thu Oct 15, 2009 4:47 pm
Guest
Grant Edwards <invalid at (no spam) invalid.invalid> wrote:
Quote:
- No root privledge to to set up
- No mucking with kernel modules

How big a hassle that last one turns out to be probably depends
on which kernel APIs you end up using. Some of them (e.g. the
tty layer) seem to be rather fluid.

The drivers are already written. One is called /dev/fanout and is
a filesystem based broadcast mechanism. I wrote fanout about four
years ago.

The other driver is /dev/proxy and is a filesystem equivalent of a
unix domain socket. Proxy may have a few features that unix sockets
do not.



Now, back to robots and their device drivers ....
Unix sockets are a hassle since 'cat' does not work immediately. We
would prefer like something like:
cat "2000" > /dev/dp/step_count
not
cat "2000" | socat - /dev/dp/step_count

But is being able to avoid socat worth the hassle of having to
compile and install the fanout and proxy modules?


I think I'm at the point where I need to ask this of our potential
customer base. This is not something you and I need to decide.


Bob Smith
 
Jan Kandziora...
Posted: Fri Oct 16, 2009 4:13 am
Guest
Bob Smith schrieb:
Quote:
OK, but this might come down to personal preference. I like the
idea of telling a stepper motor to step 2000 steps with the command:
echo "2000" > /dev/dp/step_count

Sure, it's personal preference, but I think not mine alone. The thing above

is not a commonly expected way to work with device nodes. It more
resembles /proc or /sys nodes. To a device node, nearly all users expect
they once open the node, then continously talk to it. I would also not
expect to talk in ascii, except for terminal devices.

The thing above could be implemented through FUSE, though -- "owfs" does it
that way. Google it, it pretty much does the same hardware muxer job your
program does.


Quote:
To do the equivalent with, for example, unix sockets would probably
require an API to the sockets, so a .so library to hide the details
of the socket. But not just one .so file since to be well received
I'd need to write binding for my API for lots of languages -- PHP,
Perl, ... This does not sound "minimalist" at all.

Um, you have an ascii protocol above... It's a piece of cake just sending

name+SPACE+value+LF through a TCP connection, any reasonable high-level
language could do that by printf() or similar. It would even be
network-enabled ootb.


Quote:

Jan Kandziora <jjj at (no spam) gmx.de> wrote:
Broadcast? What do you mean? Concurrent access?
Same for flow control. Please explain.
If you haven't already, would take a few minutes to read the
article at the URL in the original post.

The idea of device broadcast is not usually visible but is in the
/dev/input subsystem. An example in the article shows how to see
this. Robot sensors (my interest) also need a broadcast system.

From what I've read, it's concurrent access. The node may be opened muliple

times. Could be achieved by sockets or ptys or FUSE or CUSE or shared
memory or...

Only pipes and FIFOs do not allow concurrent access.


Quote:
I did not mention "concurrent access" but perhaps this refers to
the idea of broadcast? One writer and many concurrent readers?

As any reader must be known to the writer in your case, I would not call

it "broadcast". Broadcast means "shouting out into space". For that, it
needs a common media, e.g. shared memory or a network with broadcast
features.


Quote:

Both named pipes and pty devices allow you to write up to 4K bytes
when just one end is open. I wanted to block all writes until
both ends were open. "Flow control" to me refers to select() in
a way. I need bidirectional transfer (ruling out named pipes)
and wanted flow control to flow in both directions up to the
select() on each side. One side being the driver daemon and the
other side being the high-level robot application. I hope this
makes sense.

It does. Your previous design choices, especially the one to allow arbitrary

programs to control anything like they want to, narrow it down to that.

Kind regards

Jan
 
Jan Kandziora...
Posted: Fri Oct 16, 2009 4:42 am
Guest
Bob Smith schrieb:
Quote:

But is being able to avoid socat worth the hassle of having to
compile and install the fanout and proxy modules?

[...]

I think I'm at the point where I need to ask this of our potential
customer base. This is not something you and I need to decide.

Although not your customer (yet?), I'm an electrical engineer (and

programmer) and often work with such hardware modules.

My first rule is to avoid any vendor supplied software as most times it's
just crap (root access required for the whole system, 100% processor load
while doing nothing, doing X11 calls from a driver program etc.) Basically,
most hardware vendors do not know anything about reasonable software
design.

If avoiding vendor software is not possible, I at least try to avoid kernel
modules. The kernel is a big, single program and driver A from company AA
may require 2.6.20, while driver B from company BB requires 2.6.25. As a
customer *not* ordering 1000 Pcs of the hardware, you are out of luck.
Company AA will redo their piece of crap in a year or whenever they like
to.

Devices with binary only drivers (regardless if userspace or kernelspace)
and without a matching protocol description are outruled at all.


Your company may be the big exception from the above, but I do not know in
advance, so I apply my rules.

For kernel modules, at least *all* of the companies doing a kernel module
for my hardware must behave nice. That's very very unlikely to happen.

Kind regards

Jan
 
Bob Smith...
Posted: Fri Oct 16, 2009 12:57 pm
Guest
Jan Kandziora wrote:
Quote:
My first rule is to avoid any vendor supplied software as most times it's
just crap (root access required for the whole system, 100% processor load
while doing nothing, doing X11 calls from a driver program etc.) Basically,
most hardware vendors do not know anything about reasonable software
design.
I kind of agree with the above. My company will publish the protocol

to the board, release the daemon under a BSD license, and if we have
drivers, release them under the GPL. I was lead author and programmer
for the book "Linux Appliance Design" and I try to be aware of the kinds
of issues you mention above.


Quote:
If avoiding vendor software is not possible, I at least try to avoid kernel
modules. The kernel is a big, single program and driver A from company AA
may require 2.6.20, while driver B from company BB requires 2.6.25. As a
customer *not* ordering 1000 Pcs of the hardware, you are out of luck.
Company AA will redo their piece of crap in a year or whenever they like
to.
Again I agree. The active word in the article title is "minimalist".

There are two _really_thin_ shims for drivers. One provides a simple
broadcast mechanism and the other is the minimum needed to let two
processes talk through a device. Both are GPL'ed and MINIMAL as far
as I can make them.


Quote:
Devices with binary only drivers (regardless if userspace or kernelspace)
and without a matching protocol description are outruled at all.
We publish everything except the Verilog source that goes into the

peripherals themselves. This includes the protocol, the daemon, and
the drivers.


Quote:
For kernel modules, at least *all* of the companies doing a kernel module
for my hardware must behave nice.
I am trying to behave nice, at least as much as I know how.



Bob Smith
 
 
Page 1 of 2    Goto page 1, 2  Next
All times are GMT - 5 Hours
The time now is Mon Nov 23, 2009 6:49 am