Home
Package PSLX¶
Python Standard Library eXtension.
A standard library for job scheduling, micro services including message queue, RPC, instant messaging and monitoring, and tooling, developed by Kaifeng Chen. The library is written compatible to Python 3.7+, and is integrated upon the distributed file system Galaxy. To use the program, please install the latest protobuf compiler and gRPC tools. This document covers the APIs of PSLX provided, and is organized as follows:
Job Scheduling¶
- Running ad-hoc and scheduled jobs with operators and operator containers.
Data Storage¶
- Data storage using plain text file, proto table, sharded proto table and timestamp-based partitioner.
RPC¶
- General framework of RPC used in PSLX.
Micro Services¶
- Instant messaging services built for Slack, Rocketchat and Microsoft Teams using the PSLX RPC framework.
- Email service for sending emails through code using the PSLX RPC framework.
- Message Queue for building simple message queue using proto buffers. Part of the implementation also is shared with the PSLX rpc framework.
- Publisher/Subscriber for building simple publisher/subscriber applications.
- PSLX Frontend Flask frontend built to monitor the health of RPC servers, container status, and browser the content of protobufs.
Tools¶
- Tools provided by PSLX for galaxy-based file io, LRC caching, SQL server read/write, Mongodb tool, timestamp key-ed partitioner fetcher and watcher, function registration.
Utilities¶
- Utilities provided by PSLX for file operation, protobuf related utilities, async unittesting, timezone related utilities, yaml file io, text coloring, and credential composing, environment variables access and decorators.
Proto Schema¶
- Internal protobuf schemas.
Please also take a look at the example folder for different example implementations.