1 """Python Daemon Management -- Server functions
3 This module implements the server part of the PDM protocols. The
4 primary object of interest herein is the listen() function, which is
5 the most generic way to create PDM listeners based on user
6 configuration, and the documentation for the repl and perf classes,
7 which describes the functioning of the REPL and PERF protocols.
10 import os, sys, socket, threading, grp, select
11 import types, pprint, traceback
14 __all__ = ["repl", "perf", "listener", "unixlistener", "tcplistener", "listen"]
19 """REPL protocol handler
21 Provides a read-eval-print loop. The primary client-side interface
22 is the pdm.cli.replclient class. Clients can send arbitrary code,
23 which is compiled and run on its own thread in the server process,
24 and output responses that are echoed back to the client.
26 Each client is provided with its own module, in which the code
27 runs. The module is prepared with a function named `echo', which
28 takes a single object and pretty-prints it as part of the command
29 response. If a command can be parsed as an expression, the value
30 it evaluates to is automatically echoed to the client. If the
31 evalution of the command terminates with an exception, its
32 traceback is echoed to the client.
34 The REPL protocol is only intended for interactive usage. In order
35 to interact programmatically with the server process, see the PERF
38 def __init__(self, cl):
40 self.mod = types.ModuleType("repl")
41 self.mod.echo = self.echo
42 self.printer = pprint.PrettyPrinter(indent = 4, depth = 6)
45 def sendlines(self, text):
46 for line in text.split("\n"):
47 self.cl.send(b" " + line.encode("utf-8") + b"\n")
50 self.sendlines(self.printer.pformat(ob))
52 def command(self, cmd):
53 cmd = cmd.decode("utf-8")
56 ccode = compile(cmd, "PDM Input", "eval")
58 ccode = compile(cmd, "PDM Input", "exec")
59 exec(ccode, self.mod.__dict__)
60 self.cl.send(b"+OK\n")
62 self.echo(eval(ccode, self.mod.__dict__))
63 self.cl.send(b"+OK\n")
65 for line in traceback.format_exception(*sys.exc_info()):
66 self.cl.send(b" " + line.encode("utf-8"))
67 self.cl.send(b"+EXC\n")
69 def handle(self, buf):
76 protocols["repl"] = repl
79 """PERF protocol handler
81 The PERF protocol provides an interface for program interaction
82 with the server process. It allows limited remote interactions
83 with Python objects over a few defined interfaces.
85 All objects that wish to be available for interaction need to
86 implement a method named `pdm_protocols' which, when called with
87 no arguments, should return a list of strings, each indicating a
88 PERF interface that the object implements. For each such
89 interface, the object must implement additional methods as
92 A client can find PERF objects to interact with either by
93 specifying the name of such an object in an existing module, or by
94 using the `dir' interface, described below. Thus, to make a PERF
95 object available for clients, it needs only be bound to a global
96 variable in a module and implement the `pdm_protocols'
97 method. When requesting an object from a module, the module must
98 already be imported. PDM will not import new modules for clients;
99 rather, the daemon process needs to import all modules that
100 clients should be able to interact with. PDM itself always imports
101 the pdm.perf module, which contains a few basic PERF objects. See
102 its documentation for details.
104 The following interfaces are currently known to PERF.
107 An object that implements the `attr' interface models an
108 attribute that can be read by clients. The attribute can be
109 anything, as long as its representation can be
110 pickled. Examples of attributes could be such things as the CPU
111 time consumed by the server process, or the number of active
112 connections to whatever clients the program serves. To
113 implement the `attr' interface, an object must implement
114 methods called `readattr' and `attrinfo'. `readattr' is called
115 with no arguments to read the current value of the attribute,
116 and `attrinfo' is called with no arguments to read a
117 description of the attribute. Both should be
118 idempotent. `readattr' can return any pickleable object, and
119 `attrinfo' should return either None to indicate that it has no
120 description, or an instance of the pdm.perf.attrinfo class.
123 The `dir' interface models a directory of other PERF
124 objects. An object implementing it must implement methods
125 called `lookup' and `listdir'. `lookup' is called with a single
126 string argument that names an object, and should either return
127 another PERF object based on the name, or raise KeyError if it
128 does not recognize the name. `listdir' is called with no
129 arguments, and should return a list of known names that can be
130 used as argument to `lookup', but the list is not required to
131 be exhaustive and may also be empty.
134 The `invoke' interface allows a more arbitrary form of method
135 calls to objects implementing it. Such objects must implement a
136 method called `invoke', which is called with one positional
137 argument naming a method to be called (which it is free to
138 interpret however it wishes), and with any additional
139 positional and keyword arguments that the client wishes to pass
140 to it. Whatever `invoke' returns is pickled and sent back to
141 the client. In case the method name is not recognized, `invoke'
142 should raise an AttributeError.
145 The `event' interface allows PERF objects to notify clients of
146 events asynchronously. Objects implementing it must implement
147 methods called `subscribe' and `unsubscribe'. `subscribe' will
148 be called with a single argument, which is a callable of one
149 argument, which should be registered to be called when an event
150 pertaining to the `event' object in question occurs. The
151 `event' object should then call all such registered callables
152 with a single argument describing the event. The argument could
153 be any object that can be pickled, but should be an instance of
154 a subclass of the pdm.perf.event class. If `subscribe' is
155 called with a callback object that it has already registered,
156 it should raise a ValueError. `unsubscribe' is called with a
157 single argument, which is a previously registered callback
158 object, which should then be unregistered to that it is no
159 longer called when an event occurs. If the given callback
160 object is not, in fact, registered, a ValueError should be
163 The pdm.perf module contains a few convenience classes which
164 implements the interfaces, but PERF objects are not required to be
165 instances of them. Any object can implement a PERF interface, as
166 long as it does so as described above.
168 The pdm.cli.perfclient class is the client-side implementation.
170 def __init__(self, cl):
175 self.lock = threading.Lock()
179 for id, recv in self.subscribed.items():
181 if ob is None: continue
187 def send(self, *args):
190 buf = pickle.dumps(args)
191 buf = struct.pack(">l", len(buf)) + buf
196 def bindob(self, id, ob):
197 if not hasattr(ob, "pdm_protocols"):
198 raise ValueError("Object does not support PDM introspection")
200 proto = ob.pdm_protocols()
201 except Exception as exc:
202 raise ValueError("PDM introspection failed", exc)
203 self.odtab[id] = ob, proto
206 def bind(self, id, module, obnm):
207 resmod = sys.modules.get(module)
209 self.send("-", ImportError("No such module: %s" % module))
212 ob = getattr(resmod, obnm)
213 except AttributeError:
214 self.send("-", AttributeError("No such object: %s" % obnm))
217 proto = self.bindob(id, ob)
218 except Exception as exc:
221 self.send("+", proto)
223 def getob(self, id, proto):
224 ob = self.odtab.get(id)
226 self.send("-", ValueError("No such bound ID: %r" % id))
229 if proto not in protos:
230 self.send("-", ValueError("Object does not support that protocol"))
234 def lookup(self, tgtid, srcid, obnm):
235 src = self.getob(srcid, "dir")
239 ob = src.lookup(obnm)
240 except KeyError as exc:
244 proto = self.bindob(tgtid, ob)
245 except Exception as exc:
248 self.send("+", proto)
250 def unbind(self, id):
251 ob = self.odtab.get(id)
253 self.send("-", KeyError("No such name bound: %r" % id))
257 recv = self.subscribed.get(id)
260 del self.subscribed[id]
263 def listdir(self, id):
264 ob = self.getob(id, "dir")
267 self.send("+", ob.listdir())
269 def readattr(self, id):
270 ob = self.getob(id, "attr")
275 except Exception as exc:
276 self.send("-", Exception("Could not read attribute"))
280 def attrinfo(self, id):
281 ob = self.getob(id, "attr")
284 self.send("+", ob.attrinfo())
286 def invoke(self, id, method, args, kwargs):
287 ob = self.getob(id, "invoke")
291 self.send("+", ob.invoke(method, *args, **kwargs))
292 except Exception as exc:
295 def event(self, id, ob, ev):
296 self.send("*", id, ev)
298 def subscribe(self, id):
299 ob = self.getob(id, "event")
302 if id in self.subscribed:
303 self.send("-", ValueError("Already subscribed"))
305 self.event(id, ob, ev)
307 self.subscribed[id] = recv
310 def unsubscribe(self, id):
311 ob = self.getob(id, "event")
314 recv = self.subscribed.get(id)
316 self.send("-", ValueError("Not subscribed"))
318 del self.subscribed[id]
321 def command(self, data):
325 elif cmd == "unbind":
326 self.unbind(*data[1:])
327 elif cmd == "lookup":
328 self.lookup(*data[1:])
330 self.listdir(*data[1:])
331 elif cmd == "readattr":
332 self.readattr(*data[1:])
333 elif cmd == "attrinfo":
334 self.attrinfo(*data[1:])
335 elif cmd == "invoke":
336 self.invoke(*data[1:])
338 self.subscribe(*data[1:])
339 elif cmd == "unsubs":
340 self.unsubscribe(*data[1:])
342 self.send("-", Exception("Unknown command: %r" % (cmd,)))
344 def handle(self, buf):
347 dlen = struct.unpack(">l", buf[:4])[0]
348 if len(buf) < dlen + 4:
350 data = pickle.loads(buf[4:dlen + 4])
352 return buf[dlen + 4:]
354 protocols["perf"] = perf
356 class client(threading.Thread):
357 def __init__(self, sk):
358 super().__init__(name = "Management client")
363 def send(self, data):
364 return self.sk.send(data)
366 def choose(self, proto):
368 proto = proto.decode("ascii")
371 if proto in protocols:
372 self.handler = protocols[proto](self)
374 self.send("-ERR Unknown protocol: %s\n" % proto)
377 def handle(self, buf):
388 self.send(b"+PDM1\n")
390 ret = self.sk.recv(1024)
396 nbuf = self.handler.handle(buf)
398 #for line in traceback.format_exception(*sys.exc_info()):
408 if hasattr(self.handler, "closed"):
409 self.handler.closed()
412 class listener(threading.Thread):
415 This subclass of a thread listens to PDM connections and handles
416 client connections properly. It is intended to be subclassed by
417 providers of specific domains, such as unixlistener and
421 super().__init__(name = "Management listener")
424 def listen(self, sk):
425 """Listen for and accept connections."""
428 rfd, wfd, efd = select.select([sk], [], [sk], 1)
431 nsk, addr = sk.accept()
432 self.accept(nsk, addr)
435 """Stop listening for client connections
437 Tells the listener thread to stop listening, and then waits
443 def accept(self, sk, addr):
447 class unixlistener(listener):
448 """Unix socket listener"""
449 def __init__(self, name, mode = 0o600, group = None):
450 """Create a listener that will bind to the Unix socket named
451 by `name'. The socket will not actually be bound until the
452 listener is started. The socket will be chmodded to `mode',
453 and if `group' is given, the named group will be set as the
462 sk = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM)
465 if os.path.exists(self.name) and os.path.stat.S_ISSOCK(os.stat(self.name).st_mode):
469 os.chmod(self.name, self.mode)
470 if self.group is not None:
471 os.chown(self.name, os.getuid(), grp.getgrnam(self.group).gr_gid)
479 class tcplistener(listener):
480 """TCP socket listener"""
481 def __init__(self, port, bindaddr = "127.0.0.1"):
482 """Create a listener that will bind to the given TCP port, and
483 the given local interface. The socket will not actually be
484 bound until the listener is started.
488 self.bindaddr = bindaddr
491 sk = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
493 sk.bind((self.bindaddr, self.port))
500 """Create and start a listener according to a string
501 specification. The string specifications can easily be passed from
502 command-line options, user configuration or the like. Currently,
503 the two following specification formats are recognized:
505 PATH[:MODE[:GROUP]] -- PATH must contain at least one slash. A
506 Unix socket listener will be created listening to that path, and
507 the socket will be chmodded to MODE and owned by GROUP. If MODE is
508 not given, it defaults to 0600, and if GROUP is not given, the
509 process' default group is used.
511 ADDRESS:PORT -- PORT must be entirely numeric. A TCP socket
512 listener will be created listening to that port, bound to the
513 given local interface address. Since PDM has no authentication
514 support, ADDRESS should probably be localhost.
517 first = spec[:spec.index(":")]
518 last = spec[spec.rindex(":") + 1:]
523 parts = spec.split(":")
527 mode = int(parts[1], 8)
530 ret = unixlistener(parts[0], mode = mode, group = group)
536 port = int(spec[p + 1:])
537 ret = tcplistener(port, bindaddr = host)
540 raise ValueError("Unparsable listener specification: %r" % spec)