vieter-v
/
v
2
0
Fork 0
Code Issues Pull Requests Releases Activity

pico.v and dependencies

pull/3540/head
S-YOU 2020-01-23 11:26:30 +09:00 committed by Alexander Medvednikov
parent 5c6032d272
commit 7b345e207d
19 changed files with 2475 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

1
examples/pico/.gitignore vendored 100644
View File

@ -0,0 +1 @@
pico

41
examples/pico/pico.v 100644
View File

@ -0,0 +1,41 @@
import json
import picoev
import picohttpparser
struct Message {
message string
}
[inline]
fn json_response() string {
msg := Message{
message: 'Hello, World!'
}
return json.encode(msg)
}
[inline]
fn hello_response() string {
return 'Hello, World!'
}
pub fn callback(req picohttpparser.Request, res mut picohttpparser.Response) {
if picohttpparser.cmpn(req.method, 'GET ', 4) {
if picohttpparser.cmp(req.path, '/t') {
res.http_ok().header_server().header_date().plain().body(hello_response())
}
else if picohttpparser.cmp(req.path, '/j') {
res.http_ok().header_server().header_date().json().body(json_response())
}
else {
res.http_404()
}
}
else {
res.http_405()
}
}
pub fn main() {
picoev.new(8088, &callback).serve()
}

9
thirdparty/picoev/picoev.c vendored 100644
View File

@ -0,0 +1,9 @@
#ifdef __linux__
#include "src/picoev_epoll.c"
#elif __APPLE__
#include "src/picoev_kqueue.c"
#elif defined(__FreeBSD__) || defined(__NetBSD__) || defined(__OpenBSD__) || defined(__DragonFly__)
#include "src/picoev_kqueue.c"
#else
#include "src/picoev_select.c"
#endif

21
thirdparty/picoev/src/README.md vendored 100644
View File

@ -0,0 +1,21 @@
picoev
======
A *tiny*, *lightning fast* event loop for network applications
The text below is copied from the [original publication](http://developer.cybozu.co.jp/archives/kazuho/2009/08/picoev-a-tiny-e.html)
I am sure many programmers writing network applications have their own abstracting layers hiding the differences between various I/O multiplex APIs, like select(2), poll(2), epoll(2), ... And of course, I am one among them. While writing mycached ([see Mycached: memcached protocol support for MySQL for more information](http://developer.cybozu.co.jp/archives/kazuho/2009/08/mycached-memcac.html)), I was at first considering of using [libev](http://software.schmorp.de/pkg/libev.html) for multiplexing socket I/Os. [Libevent](http://www.monkey.org/~provos/libevent/) was not an option since it does not (yet) provide multithreading support.
But it was a great pain for me to learn how to use libev. I do not mean that its is an ugly product. In fact, I think that it is a very well written, excellent library. However, for me it was too much a boring task to learn how the things are abstracted, already being familiar with the problems it tries to hide.
So instead I thought it might be a good occasion to write my own library that could be used in any programs I may write in the future. The result is picoev, and it is faster than libevent or libev! The benchmark used is a re-modified version taken from libev.schmorp.de/bench.html and can be found [here](http://coderepos.org/share/browser/lang/c/picoev/trunk/example/bench.c).
![setup time](http://developer.cybozu.co.jp/archives/kazuho/files/picoev_setup.png)
![event processing time](http://developer.cybozu.co.jp/archives/kazuho/files/picoev_event.png)
Why is it faster? It is because it uses an array and a ring buffer of bit vectors as its internal structure. Libevent and libev seem to use some kind of sorted tree to represent file descriptors. However, if we concentrate on Un*x systems, there is a guarantee that the descriptors will be a small positive integer. Picoev utilizes the fact and stores information related to file descriptors (such as pointers to callback functions or callback arguments) in an array, resulting in a faster manipulation of socket states.
Another optimization technique used by picoev is not to use an ordered tree for keeping timeout information. Generally speaking, most network applications do not require accurate timeouts. Thus it is possible to use a ring buffer (a sliding array) of bit vectors for the purpose. Each bit vector represents a set of file descriptors that time-outs at a given time. Picoev uses 128 of bit vectors to represent timeouts, for example, the first bit vector represents the sockets that timeout a second after, the second bit vector representing them of two seconds after..., and the bit vectors slide every second. If the maximum timeout required by the web application is greater than 128, the minimum granurality of timeout becomes two seconds.
I would like to reiterate that both libevent and libev are great libraries. Picoev is not at comparable to them especially in maturity and the number of features. It only supports select(2), epoll(2), and kqueue(2) for the time being. However the design is simple than the other two, and I think it will be a good starting point to write network applications, or to use as a basis for writing one's own network libraries.

404
thirdparty/picoev/src/picoev.h vendored 100644
View File

@ -0,0 +1,404 @@
/*
* Copyright (c) 2009, Cybozu Labs, Inc.
* All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions are met:
*
* * Redistributions of source code must retain the above copyright notice,
* this list of conditions and the following disclaimer.
* * Redistributions in binary form must reproduce the above copyright notice,
* this list of conditions and the following disclaimer in the documentation
* and/or other materials provided with the distribution.
* * Neither the name of the <ORGANIZATION> nor the names of its contributors
* may be used to endorse or promote products derived from this software
* without specific prior written permission.
*
* THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
* AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
* IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
* ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE
* LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
* CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
* SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
* INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
* CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
* ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
* POSSIBILITY OF SUCH DAMAGE.
*/
#ifndef picoev_h
#define picoev_h
#ifdef __cplusplus
extern "C" {
# define PICOEV_INLINE inline
#else
# define PICOEV_INLINE static __inline__
#endif
#include <assert.h>
#include <limits.h>
#include <stdlib.h>
#include <string.h>
#include <time.h>
#define PICOEV_IS_INITED (picoev.max_fd != 0)
#define PICOEV_IS_INITED_AND_FD_IN_RANGE(fd) \
(((unsigned)fd) < (unsigned)picoev.max_fd)
#define PICOEV_TOO_MANY_LOOPS (picoev.num_loops != 0) /* use after ++ */
#define PICOEV_FD_BELONGS_TO_LOOP(loop, fd) \
((loop)->loop_id == picoev.fds[fd].loop_id)
#define PICOEV_TIMEOUT_VEC_OF(loop, idx) \
((loop)->timeout.vec + (idx) * picoev.timeout_vec_size)
#define PICOEV_TIMEOUT_VEC_OF_VEC_OF(loop, idx) \
((loop)->timeout.vec_of_vec + (idx) * picoev.timeout_vec_of_vec_size)
#define PICOEV_RND_UP(v, d) (((v) + (d) - 1) / (d) * (d))
#define PICOEV_PAGE_SIZE 4096
#define PICOEV_CACHE_LINE_SIZE 32 /* in bytes, ok if greater than the actual */
#define PICOEV_SIMD_BITS 128
#define PICOEV_TIMEOUT_VEC_SIZE 128
#define PICOEV_SHORT_BITS (sizeof(short) * 8)
#define PICOEV_READ 1
#define PICOEV_WRITE 2
#define PICOEV_TIMEOUT 4
#define PICOEV_ADD 0x40000000
#define PICOEV_DEL 0x20000000
#define PICOEV_READWRITE (PICOEV_READ | PICOEV_WRITE)
#define PICOEV_TIMEOUT_IDX_UNUSED (UCHAR_MAX)
typedef unsigned short picoev_loop_id_t;
typedef struct picoev_loop_st picoev_loop;
typedef void picoev_handler(picoev_loop* loop, int fd, int revents,
void* cb_arg);
typedef struct picoev_fd_st {
/* use accessors! */
/* TODO adjust the size to match that of a cache line */
picoev_handler* callback;
void* cb_arg;
picoev_loop_id_t loop_id;
char events;
unsigned char timeout_idx; /* PICOEV_TIMEOUT_IDX_UNUSED if not used */
int _backend; /* can be used by backends (never modified by core) */
} picoev_fd;
struct picoev_loop_st {
/* read only */
picoev_loop_id_t loop_id;
struct {
short* vec;
short* vec_of_vec;
size_t base_idx;
time_t base_time;
int resolution;
void* _free_addr;
} timeout;
time_t now;
};
typedef struct picoev_globals_st {
/* read only */
picoev_fd* fds;
void* _fds_free_addr;
int max_fd;
int num_loops;
size_t timeout_vec_size; /* # of elements in picoev_loop.timeout.vec[0] */
size_t timeout_vec_of_vec_size; /* ... in timeout.vec_of_vec[0] */
} picoev_globals;
extern picoev_globals picoev;
/* creates a new event loop (defined by each backend) */
picoev_loop* picoev_create_loop(int max_timeout);
/* destroys a loop (defined by each backend) */
int picoev_destroy_loop(picoev_loop* loop);
/* internal: updates events to be watched (defined by each backend) */
int picoev_update_events_internal(picoev_loop* loop, int fd, int events);
/* internal: poll once and call the handlers (defined by each backend) */
int picoev_poll_once_internal(picoev_loop* loop, int max_wait);
/* internal, aligned allocator with address scrambling to avoid cache
line contention */
PICOEV_INLINE
void* picoev_memalign(size_t sz, void** orig_addr, int clear) {
sz = sz + PICOEV_PAGE_SIZE + PICOEV_CACHE_LINE_SIZE;
if ((*orig_addr = malloc(sz)) == NULL) {
return NULL;
}
if (clear != 0) {
memset(*orig_addr, 0, sz);
}
return
(void*)PICOEV_RND_UP((unsigned long)*orig_addr
+ (rand() % PICOEV_PAGE_SIZE),
PICOEV_CACHE_LINE_SIZE);
}
/* initializes picoev */
PICOEV_INLINE
int picoev_init(int max_fd) {
assert(! PICOEV_IS_INITED);
assert(max_fd > 0);
if ((picoev.fds = (picoev_fd*)picoev_memalign(sizeof(picoev_fd) * max_fd,
&picoev._fds_free_addr, 1))
== NULL) {
return -1;
}
picoev.max_fd = max_fd;
picoev.num_loops = 0;
picoev.timeout_vec_size
= PICOEV_RND_UP(picoev.max_fd, PICOEV_SIMD_BITS) / PICOEV_SHORT_BITS;
picoev.timeout_vec_of_vec_size
= PICOEV_RND_UP(picoev.timeout_vec_size, PICOEV_SIMD_BITS)
/ PICOEV_SHORT_BITS;
return 0;
}
/* deinitializes picoev */
PICOEV_INLINE
int picoev_deinit(void) {
assert(PICOEV_IS_INITED);
free(picoev._fds_free_addr);
picoev.fds = NULL;
picoev._fds_free_addr = NULL;
picoev.max_fd = 0;
picoev.num_loops = 0;
return 0;
}
/* updates timeout */
PICOEV_INLINE
void picoev_set_timeout(picoev_loop* loop, int fd, int secs) {
picoev_fd* target;
short* vec, * vec_of_vec;
size_t vi = fd / PICOEV_SHORT_BITS, delta;
assert(PICOEV_IS_INITED_AND_FD_IN_RANGE(fd));
assert(PICOEV_FD_BELONGS_TO_LOOP(loop, fd));
target = picoev.fds + fd;
/* clear timeout */
if (target->timeout_idx != PICOEV_TIMEOUT_IDX_UNUSED) {
vec = PICOEV_TIMEOUT_VEC_OF(loop, target->timeout_idx);
if ((vec[vi] &= ~((unsigned short)SHRT_MIN >> (fd % PICOEV_SHORT_BITS)))
== 0) {
vec_of_vec = PICOEV_TIMEOUT_VEC_OF_VEC_OF(loop, target->timeout_idx);
vec_of_vec[vi / PICOEV_SHORT_BITS]
&= ~((unsigned short)SHRT_MIN >> (vi % PICOEV_SHORT_BITS));
}
target->timeout_idx = PICOEV_TIMEOUT_IDX_UNUSED;
}
if (secs != 0) {
delta = (loop->now + secs - loop->timeout.base_time)
/ loop->timeout.resolution;
if (delta >= PICOEV_TIMEOUT_VEC_SIZE) {
delta = PICOEV_TIMEOUT_VEC_SIZE - 1;
}
target->timeout_idx =
(loop->timeout.base_idx + delta) % PICOEV_TIMEOUT_VEC_SIZE;
vec = PICOEV_TIMEOUT_VEC_OF(loop, target->timeout_idx);
vec[vi] |= (unsigned short)SHRT_MIN >> (fd % PICOEV_SHORT_BITS);
vec_of_vec = PICOEV_TIMEOUT_VEC_OF_VEC_OF(loop, target->timeout_idx);
vec_of_vec[vi / PICOEV_SHORT_BITS]
|= (unsigned short)SHRT_MIN >> (vi % PICOEV_SHORT_BITS);
}
}
/* registers a file descriptor and callback argument to a event loop */
PICOEV_INLINE
int picoev_add(picoev_loop* loop, int fd, int events, int timeout_in_secs,
picoev_handler* callback, void* cb_arg) {
picoev_fd* target;
if (!PICOEV_IS_INITED_AND_FD_IN_RANGE(fd)) { return -1; }
target = picoev.fds + fd;
assert(target->loop_id == 0);
target->callback = callback;
target->cb_arg = cb_arg;
target->loop_id = loop->loop_id;
target->events = 0;
target->timeout_idx = PICOEV_TIMEOUT_IDX_UNUSED;
if (picoev_update_events_internal(loop, fd, events | PICOEV_ADD) != 0) {
target->loop_id = 0;
return -1;
}
picoev_set_timeout(loop, fd, timeout_in_secs);
return 0;
}
/* unregisters a file descriptor from event loop */
PICOEV_INLINE
int picoev_del(picoev_loop* loop, int fd) {
picoev_fd* target;
assert(PICOEV_IS_INITED_AND_FD_IN_RANGE(fd));
target = picoev.fds + fd;
if (picoev_update_events_internal(loop, fd, PICOEV_DEL) != 0) {
return -1;
}
picoev_set_timeout(loop, fd, 0);
target->loop_id = 0;
return 0;
}
/* check if fd is registered (checks all loops if loop == NULL) */
PICOEV_INLINE
int picoev_is_active(picoev_loop* loop, int fd) {
assert(PICOEV_IS_INITED_AND_FD_IN_RANGE(fd));
return loop != NULL
? picoev.fds[fd].loop_id == loop->loop_id
: picoev.fds[fd].loop_id != 0;
}
/* returns events being watched for given descriptor */
PICOEV_INLINE
int picoev_get_events(picoev_loop* loop __attribute__((unused)), int fd) {
assert(PICOEV_IS_INITED_AND_FD_IN_RANGE(fd));
return picoev.fds[fd].events & PICOEV_READWRITE;
}
/* sets events to be watched for given desriptor */
PICOEV_INLINE
int picoev_set_events(picoev_loop* loop, int fd, int events) {
assert(PICOEV_IS_INITED_AND_FD_IN_RANGE(fd));
if (picoev.fds[fd].events != events
&& picoev_update_events_internal(loop, fd, events) != 0) {
return -1;
}
return 0;
}
/* returns callback for given descriptor */
PICOEV_INLINE
picoev_handler* picoev_get_callback(picoev_loop* loop __attribute__((unused)),
int fd, void** cb_arg) {
assert(PICOEV_IS_INITED_AND_FD_IN_RANGE(fd));
if (cb_arg != NULL) {
*cb_arg = picoev.fds[fd].cb_arg;
}
return picoev.fds[fd].callback;
}
/* sets callback for given descriptor */
PICOEV_INLINE
void picoev_set_callback(picoev_loop* loop __attribute__((unused)), int fd,
picoev_handler* callback, void** cb_arg) {
assert(PICOEV_IS_INITED_AND_FD_IN_RANGE(fd));
if (cb_arg != NULL) {
picoev.fds[fd].cb_arg = *cb_arg;
}
picoev.fds[fd].callback = callback;
}
/* function to iterate registered information. To start iteration, set curfd
to -1 and call the function until -1 is returned */
PICOEV_INLINE
int picoev_next_fd(picoev_loop* loop, int curfd) {
if (curfd != -1) {
assert(PICOEV_IS_INITED_AND_FD_IN_RANGE(curfd));
}
while (++curfd < picoev.max_fd) {
if (loop->loop_id == picoev.fds[curfd].loop_id) {
return curfd;
}
}
return -1;
}
/* internal function */
PICOEV_INLINE
int picoev_init_loop_internal(picoev_loop* loop, int max_timeout) {
loop->loop_id = ++picoev.num_loops;
assert(PICOEV_TOO_MANY_LOOPS);
if ((loop->timeout.vec_of_vec
= (short*)picoev_memalign((picoev.timeout_vec_of_vec_size
+ picoev.timeout_vec_size)
* sizeof(short) * PICOEV_TIMEOUT_VEC_SIZE,
&loop->timeout._free_addr, 1))
== NULL) {
--picoev.num_loops;
return -1;
}
loop->timeout.vec = loop->timeout.vec_of_vec
+ picoev.timeout_vec_of_vec_size * PICOEV_TIMEOUT_VEC_SIZE;
loop->timeout.base_idx = 0;
loop->timeout.base_time = time(NULL);
loop->timeout.resolution
= PICOEV_RND_UP(max_timeout, PICOEV_TIMEOUT_VEC_SIZE)
/ PICOEV_TIMEOUT_VEC_SIZE;
return 0;
}
/* internal function */
PICOEV_INLINE
void picoev_deinit_loop_internal(picoev_loop* loop) {
free(loop->timeout._free_addr);
}
/* internal function */
PICOEV_INLINE
void picoev_handle_timeout_internal(picoev_loop* loop) {
size_t i, j, k;
for (;
loop->timeout.base_time <= loop->now - loop->timeout.resolution;
loop->timeout.base_idx
= (loop->timeout.base_idx + 1) % PICOEV_TIMEOUT_VEC_SIZE,
loop->timeout.base_time += loop->timeout.resolution) {
/* TODO use SIMD instructions */
short* vec = PICOEV_TIMEOUT_VEC_OF(loop, loop->timeout.base_idx);
short* vec_of_vec
= PICOEV_TIMEOUT_VEC_OF_VEC_OF(loop, loop->timeout.base_idx);
for (i = 0; i < picoev.timeout_vec_of_vec_size; ++i) {
short vv = vec_of_vec[i];
if (vv != 0) {
for (j = i * PICOEV_SHORT_BITS; vv != 0; j++, vv <<= 1) {
if (vv < 0) {
short v = vec[j];
assert(v != 0);
for (k = j * PICOEV_SHORT_BITS; v != 0; k++, v <<= 1) {
if (v < 0) {
picoev_fd* fd = picoev.fds + k;
assert(fd->loop_id == loop->loop_id);
fd->timeout_idx = PICOEV_TIMEOUT_IDX_UNUSED;
(*fd->callback)(loop, k, PICOEV_TIMEOUT, fd->cb_arg);
}
}
vec[j] = 0;
}
}
vec_of_vec[i] = 0;
}
}
}
}
/* loop once */
PICOEV_INLINE
int picoev_loop_once(picoev_loop* loop, int max_wait) {
loop->now = time(NULL);
if (max_wait > loop->timeout.resolution) {
max_wait = loop->timeout.resolution;
}
if (picoev_poll_once_internal(loop, max_wait) != 0) {
return -1;
}
if (max_wait != 0) {
loop->now = time(NULL);
}
picoev_handle_timeout_internal(loop);
return 0;
}
#undef PICOEV_INLINE
#ifdef __cplusplus
}
#endif
#endif

168
thirdparty/picoev/src/picoev_epoll.c vendored 100644
View File

@ -0,0 +1,168 @@
/*
* Copyright (c) 2009, Cybozu Labs, Inc.
* All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions are met:
*
* * Redistributions of source code must retain the above copyright notice,
* this list of conditions and the following disclaimer.
* * Redistributions in binary form must reproduce the above copyright notice,
* this list of conditions and the following disclaimer in the documentation
* and/or other materials provided with the distribution.
* * Neither the name of the <ORGANIZATION> nor the names of its contributors
* may be used to endorse or promote products derived from this software
* without specific prior written permission.
*
* THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
* AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
* IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
* ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE
* LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
* CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
* SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
* INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
* CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
* ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
* POSSIBILITY OF SUCH DAMAGE.
*/
#include <errno.h>
#include <sys/epoll.h>
#include <unistd.h>
#include "picoev.h"
#ifndef PICOEV_EPOLL_DEFER_DELETES
# define PICOEV_EPOLL_DEFER_DELETES 1
#endif
typedef struct picoev_loop_epoll_st {
picoev_loop loop;
int epfd;
struct epoll_event events[1024];
} picoev_loop_epoll;
picoev_globals picoev;
picoev_loop* picoev_create_loop(int max_timeout)
{
picoev_loop_epoll* loop;
/* init parent */
assert(PICOEV_IS_INITED);
if ((loop = (picoev_loop_epoll*)malloc(sizeof(picoev_loop_epoll))) == NULL) {
return NULL;
}
if (picoev_init_loop_internal(&loop->loop, max_timeout) != 0) {
free(loop);
return NULL;
}
/* init myself */
if ((loop->epfd = epoll_create(picoev.max_fd)) == -1) {
picoev_deinit_loop_internal(&loop->loop);
free(loop);
return NULL;
}
loop->loop.now = time(NULL);
return &loop->loop;
}
int picoev_destroy_loop(picoev_loop* _loop)
{
picoev_loop_epoll* loop = (picoev_loop_epoll*)_loop;
if (close(loop->epfd) != 0) {
return -1;
}
picoev_deinit_loop_internal(&loop->loop);
free(loop);
return 0;
}
int picoev_update_events_internal(picoev_loop* _loop, int fd, int events)
{
picoev_loop_epoll* loop = (picoev_loop_epoll*)_loop;
picoev_fd* target = picoev.fds + fd;
struct epoll_event ev;
int epoll_ret;
memset( &ev, 0, sizeof( ev ) );
assert(PICOEV_FD_BELONGS_TO_LOOP(&loop->loop, fd));
if ((events & PICOEV_READWRITE) == target->events) {
return 0;
}
ev.events = ((events & PICOEV_READ) != 0 ? EPOLLIN : 0)
| ((events & PICOEV_WRITE) != 0 ? EPOLLOUT : 0);
ev.data.fd = fd;
#define SET(op, check_error) do { \
epoll_ret = epoll_ctl(loop->epfd, op, fd, &ev); \
assert(! check_error || epoll_ret == 0); \
} while (0)
#if PICOEV_EPOLL_DEFER_DELETES
if ((events & PICOEV_DEL) != 0) {
/* nothing to do */
} else if ((events & PICOEV_READWRITE) == 0) {
SET(EPOLL_CTL_DEL, 1);
} else {
SET(EPOLL_CTL_MOD, 0);
if (epoll_ret != 0) {
assert(errno == ENOENT);
SET(EPOLL_CTL_ADD, 1);
}
}
#else
if ((events & PICOEV_READWRITE) == 0) {
SET(EPOLL_CTL_DEL, 1);
} else {
SET(target->events == 0 ? EPOLL_CTL_ADD : EPOLL_CTL_MOD, 1);
}
#endif
#undef SET
target->events = events;
return 0;
}
int picoev_poll_once_internal(picoev_loop* _loop, int max_wait)
{
picoev_loop_epoll* loop = (picoev_loop_epoll*)_loop;
int i, nevents;
nevents = epoll_wait(loop->epfd, loop->events,
sizeof(loop->events) / sizeof(loop->events[0]),
max_wait * 1000);
if (nevents == -1) {
return -1;
}
for (i = 0; i < nevents; ++i) {
struct epoll_event* event = loop->events + i;
picoev_fd* target = picoev.fds + event->data.fd;
if (loop->loop.loop_id == target->loop_id
&& (target->events & PICOEV_READWRITE) != 0) {
int revents = ((event->events & EPOLLIN) != 0 ? PICOEV_READ : 0)
| ((event->events & EPOLLOUT) != 0 ? PICOEV_WRITE : 0);
if (revents != 0) {
(*target->callback)(&loop->loop, event->data.fd, revents,
target->cb_arg);
}
} else {
#if PICOEV_EPOLL_DEFER_DELETES
event->events = 0;
epoll_ctl(loop->epfd, EPOLL_CTL_DEL, event->data.fd, event);
#endif
}
}
return 0;
}

209
thirdparty/picoev/src/picoev_kqueue.c vendored 100644
View File

@ -0,0 +1,209 @@
/*
* Copyright (c) 2009, Cybozu Labs, Inc.
* All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions are met:
*
* * Redistributions of source code must retain the above copyright notice,
* this list of conditions and the following disclaimer.
* * Redistributions in binary form must reproduce the above copyright notice,
* this list of conditions and the following disclaimer in the documentation
* and/or other materials provided with the distribution.
* * Neither the name of the <ORGANIZATION> nor the names of its contributors
* may be used to endorse or promote products derived from this software
* without specific prior written permission.
*
* THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
* AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
* IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
* ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE
* LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
* CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
* SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
* INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
* CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
* ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
* POSSIBILITY OF SUCH DAMAGE.
*/
#include <errno.h>
#include <sys/types.h>
#include <sys/event.h>
#include <sys/time.h>
#include <unistd.h>
#include "picoev.h"
#define EV_QUEUE_SZ 128
#define BACKEND_BUILD(next_fd, events) \
((unsigned)((next_fd << 8) | (events & 0xff)))
#define BACKEND_GET_NEXT_FD(backend) ((int)(backend) >> 8)
#define BACKEND_GET_OLD_EVENTS(backend) ((int)(backend) & 0xff)
typedef struct picoev_loop_kqueue_st {
picoev_loop loop;
int kq;
int changed_fds; /* link list using picoev_fd::_backend, -1 if not changed */
struct kevent events[1024];
struct kevent changelist[256];
} picoev_loop_kqueue;
picoev_globals picoev;
static int apply_pending_changes(picoev_loop_kqueue* loop, int apply_all)
{
#define SET(op, events) \
EV_SET(loop->changelist + cl_off++, loop->changed_fds, \
(((events) & PICOEV_READ) != 0 ? EVFILT_READ : 0) \
| (((events) & PICOEV_WRITE) != 0 ? EVFILT_WRITE : 0), \
(op), 0, 0, NULL)
int cl_off = 0, nevents;
while (loop->changed_fds != -1) {
picoev_fd* changed = picoev.fds + loop->changed_fds;
int old_events = BACKEND_GET_OLD_EVENTS(changed->_backend);
if (changed->events != old_events) {
if (old_events != 0) {
SET(EV_DISABLE, old_events);
}
if (changed->events != 0) {
SET(EV_ADD | EV_ENABLE, changed->events);
}
if ((size_t)cl_off + 1
>= sizeof(loop->changelist) / sizeof(loop->changelist[0])) {
nevents = kevent(loop->kq, loop->changelist, cl_off, NULL, 0, NULL);
assert(nevents == 0);
cl_off = 0;
}
}
loop->changed_fds = BACKEND_GET_NEXT_FD(changed->_backend);
changed->_backend = -1;
}
if (apply_all && cl_off != 0) {
nevents = kevent(loop->kq, loop->changelist, cl_off, NULL, 0, NULL);
assert(nevents == 0);
cl_off = 0;
}
return cl_off;
#undef SET
}
picoev_loop* picoev_create_loop(int max_timeout)
{
picoev_loop_kqueue* loop;
/* init parent */
assert(PICOEV_IS_INITED);
if ((loop = (picoev_loop_kqueue*)malloc(sizeof(picoev_loop_kqueue)))
== NULL) {
return NULL;
}
if (picoev_init_loop_internal(&loop->loop, max_timeout) != 0) {
free(loop);
return NULL;
}
/* init kqueue */
if ((loop->kq = kqueue()) == -1) {
picoev_deinit_loop_internal(&loop->loop);
free(loop);
return NULL;
}
loop->changed_fds = -1;
loop->loop.now = time(NULL);
return &loop->loop;
}
int picoev_destroy_loop(picoev_loop* _loop)
{
picoev_loop_kqueue* loop = (picoev_loop_kqueue*)_loop;
if (close(loop->kq) != 0) {
return -1;
}
picoev_deinit_loop_internal(&loop->loop);
free(loop);
return 0;
}
int picoev_update_events_internal(picoev_loop* _loop, int fd, int events)
{
picoev_loop_kqueue* loop = (picoev_loop_kqueue*)_loop;
picoev_fd* target = picoev.fds + fd;
assert(PICOEV_FD_BELONGS_TO_LOOP(&loop->loop, fd));
/* initialize if adding the fd */
if ((events & PICOEV_ADD) != 0) {
target->_backend = -1;
}
/* return if nothing to do */
if (events == PICOEV_DEL
? target->_backend == -1
: (events & PICOEV_READWRITE) == target->events) {
return 0;
}
/* add to changed list if not yet being done */
if (target->_backend == -1) {
target->_backend = BACKEND_BUILD(loop->changed_fds, target->events);
loop->changed_fds = fd;
}
/* update events */
target->events = events & PICOEV_READWRITE;
/* apply immediately if is a DELETE */
if ((events & PICOEV_DEL) != 0) {
apply_pending_changes(loop, 1);
}
return 0;
}
int picoev_poll_once_internal(picoev_loop* _loop, int max_wait)
{
picoev_loop_kqueue* loop = (picoev_loop_kqueue*)_loop;
struct timespec ts;
int cl_off = 0, nevents, i;
/* apply pending changes, with last changes stored to loop->changelist */
cl_off = apply_pending_changes(loop, 0);
ts.tv_sec = max_wait;
ts.tv_nsec = 0;
nevents = kevent(loop->kq, loop->changelist, cl_off, loop->events,
sizeof(loop->events) / sizeof(loop->events[0]), &ts);
if (nevents == -1) {
/* the errors we can only rescue */
assert(errno == EACCES || errno == EFAULT || errno == EINTR);
return -1;
}
for (i = 0; i < nevents; ++i) {
struct kevent* event = loop->events + i;
picoev_fd* target = picoev.fds + event->ident;
assert((event->flags & EV_ERROR) == 0); /* changelist errors are fatal */
if (loop->loop.loop_id == target->loop_id
&& (event->filter & (EVFILT_READ | EVFILT_WRITE)) != 0) {
int revents;
switch (event->filter) {
case EVFILT_READ:
revents = PICOEV_READ;
break;
case EVFILT_WRITE:
revents = PICOEV_WRITE;
break;
default:
assert(0);
revents = 0; // suppress compiler warning
break;
}
(*target->callback)(&loop->loop, event->ident, revents, target->cb_arg);
}
}
return 0;
}

169
thirdparty/picoev/src/picoev_select.c vendored 100644
View File

@ -0,0 +1,169 @@
/*
* Copyright (c) 2009, Cybozu Labs, Inc.
* All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions are met:
*
* * Redistributions of source code must retain the above copyright notice,
* this list of conditions and the following disclaimer.
* * Redistributions in binary form must reproduce the above copyright notice,
* this list of conditions and the following disclaimer in the documentation
* and/or other materials provided with the distribution.
* * Neither the name of the <ORGANIZATION> nor the names of its contributors
* may be used to endorse or promote products derived from this software
* without specific prior written permission.
*
* THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
* AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
* IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
* ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE
* LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
* CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
* SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
* INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
* CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
* ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
* POSSIBILITY OF SUCH DAMAGE.
*/
#ifndef _WIN32
# include <sys/select.h>
#else
# include <ws2tcpip.h>
#endif
#include "picoev.h"
#ifdef _WIN32
# define PICOEV_W32_INTERNAL
# include "picoev_w32.h"
# define PICOEV_FD_SET(x, y) FD_SET(picoev_w32_fd2sock(x), y)
# define PICOEV_FD_ISSET(x, y) FD_ISSET(picoev_w32_fd2sock(x), y)
typedef struct picoev_w32_globals_st {
int* fds;
void* _fds_free_addr;
} picoev_w32_globals;
picoev_w32_globals picoev_w32;
int picoev_w32_sock2fd(int sock) {
int i;
for (i = 0; i < picoev.max_fd && picoev_w32.fds[i]; ++i)
if (picoev_w32.fds[i] == sock) return i;
assert(PICOEV_IS_INITED_AND_FD_IN_RANGE(i));
picoev_w32.fds[i] = sock;
return i;
}
int picoev_w32_fd2sock(int fd) {
assert(PICOEV_IS_INITED_AND_FD_IN_RANGE(fd));
return picoev_w32.fds[fd];
}
extern int picoev_w32_deinit(void);
int picoev_w32_init(int max_fd) {
int r = picoev_init(max_fd);
if ((picoev_w32.fds = (int*)picoev_memalign(sizeof(int) * max_fd,
&picoev_w32._fds_free_addr, 1))
== NULL) {
picoev_deinit();
return -1;
}
}
int picoev_w32_deinit(void) {
free(picoev_w32._fds_free_addr);
picoev_w32.fds = NULL;
picoev_w32._fds_free_addr = NULL;
return picoev_deinit();
}
#else
# define PICOEV_FD_SET(x, y) FD_SET(x, y)
# define PICOEV_FD_ISSET(x, y) FD_ISSET(x, y)
#endif
picoev_globals picoev;
picoev_loop* picoev_create_loop(int max_timeout)
{
picoev_loop* loop;
assert(PICOEV_IS_INITED);
if ((loop = (picoev_loop*)malloc(sizeof(picoev_loop))) == NULL) {
return NULL;
}
if (picoev_init_loop_internal(loop, max_timeout) != 0) {
free(loop);
return NULL;
}
loop->now = time(NULL);
return loop;
}
int picoev_destroy_loop(picoev_loop* loop)
{
picoev_deinit_loop_internal(loop);
free(loop);
return 0;
}
int picoev_update_events_internal(picoev_loop* loop, int fd, int events)
{
picoev.fds[fd].events = events & PICOEV_READWRITE;
return 0;
}
int picoev_poll_once_internal(picoev_loop* loop, int max_wait)
{
fd_set readfds, writefds, errorfds;
struct timeval tv;
int i, r, maxfd = 0;
/* setup */
FD_ZERO(&readfds);
FD_ZERO(&writefds);
FD_ZERO(&errorfds);
for (i = 0; i < picoev.max_fd; ++i) {
picoev_fd* fd = picoev.fds + i;
if (fd->loop_id == loop->loop_id) {
if ((fd->events & PICOEV_READ) != 0) {
PICOEV_FD_SET(i, &readfds);
if (maxfd < i) {
maxfd = i;
}
}
if ((fd->events & PICOEV_WRITE) != 0) {
PICOEV_FD_SET(i, &writefds);
if (maxfd < i) {
maxfd = i;
}
}
}
}
/* select and handle if any */
tv.tv_sec = max_wait;
tv.tv_usec = 0;
r = select(maxfd + 1, &readfds, &writefds, &errorfds, &tv);
if (r == -1) {
return -1;
} else if (r > 0) {
for (i = 0; i < picoev.max_fd; ++i) {
picoev_fd* target = picoev.fds + i;
if (target->loop_id == loop->loop_id) {
int revents = (PICOEV_FD_ISSET(i, &readfds) ? PICOEV_READ : 0)
| (PICOEV_FD_ISSET(i, &writefds) ? PICOEV_WRITE : 0);
if (revents != 0) {
(*target->callback)(loop, i, revents, target->cb_arg);
}
}
}
}
return 0;
}

15
thirdparty/picoev/src/picoev_w32.h vendored 100644
View File

@ -0,0 +1,15 @@
#ifndef picoev_w32_h
#define picoev_w32_h
#include "picoev.h"
#ifndef PICOEV_W32_INTERNAL
extern int picoev_w32_init(int);
extern int picoev_w32_deinit(void);
extern int picoev_w32_sock2fd(int);
extern int picoev_w32_fd2sock(int);
# define picoev_init picoev_w32_init
# define picoev_deinit picoev_w32_deinit
#endif
#endif

28
thirdparty/picohttpparser/picohttpparser.c vendored 100644
View File

@ -0,0 +1,28 @@
#include "src/picohttpparser.c"
#if !defined(__WINDOWS__) && (defined(WIN32) || defined(WIN64) || defined(_MSC_VER) || defined(_WIN32))
#define __WINDOWS__
#endif
// date
#include <time.h>
const char* get_date() {
time_t t;
struct tm tm;
static const char *days[] = {"Sun", "Mon", "Tue", "Wed", "Thu", "Fri", "Sat"};
static const char *months[] = {"Jan", "Feb", "Mar", "Apr", "May", "Jun", "Jul", "Aug", "Sep", "Oct", "Nov", "Dec"};
static char date[30] = "Thu, 01 Jan 1970 00:00:00 GMT";
time(&t);
#ifdef __WINDOWS__
gmtime_s(&t, &tm);
#else
gmtime_r(&t, &tm);
#endif
strftime(date, 30, "---, %d --- %Y %H:%M:%S GMT", &tm);
memcpy(date, days[tm.tm_wday], 3);
memcpy(date + 8, months[tm.tm_mon], 3);
return date;
}

112
thirdparty/picohttpparser/picohttpparser.h vendored 100644
View File

@ -0,0 +1,112 @@
typedef struct phr_header phr_header_t;
#include "src/picohttpparser.h"
#include <stdint.h>
#define ADVANCE_TOKEN2(buf, tok, toklen, max_len) \
do {\
for (int i = 0; i < max_len; i++) {\
if (buf[i] == ' ') {\
tok = buf;\
toklen = i++;\
while (buf[i] == ' ') i++;\
buf += i;\
break;\
}\
}\
} while (0)
static inline int phr_parse_request_path(const char *buf_start, size_t len, const char **method, size_t *method_len, const char **path, size_t *path_len) {
if (len < 14) return -2;
const char *buf = buf_start, *buf_end = buf_start + len;
ADVANCE_TOKEN2(buf, *method, *method_len, 9);
len -= buf-buf_start;
ADVANCE_TOKEN2(buf, *path, *path_len, len);
if (*method_len == 0 || *path_len == 0) return -1;
return buf-buf_start;
}
static inline int phr_parse_request_path_pipeline(const char *buf_start, size_t len, const char **method, size_t *method_len, const char **path, size_t *path_len) {
if (len < 14) return -2;
const char *buf = buf_start, *buf_end = buf_start + len;
ADVANCE_TOKEN2(buf, *method, *method_len, 9);
len -= buf-buf_start;
ADVANCE_TOKEN2(buf, *path, *path_len, len);
if (*method_len == 0 || *path_len == 0) return -1;
while (buf < buf_end) {
++buf;
if (*(uint32_t*)buf == 0x0a0d0a0d) {
buf += 4;
return (int)(buf - buf_start);
}
};
return -1;
}
// date
const char* get_date();
// branchlut
const char gDigitsLut[200] = {
'0','0','0','1','0','2','0','3','0','4','0','5','0','6','0','7','0','8','0','9',
'1','0','1','1','1','2','1','3','1','4','1','5','1','6','1','7','1','8','1','9',
'2','0','2','1','2','2','2','3','2','4','2','5','2','6','2','7','2','8','2','9',
'3','0','3','1','3','2','3','3','3','4','3','5','3','6','3','7','3','8','3','9',
'4','0','4','1','4','2','4','3','4','4','4','5','4','6','4','7','4','8','4','9',
'5','0','5','1','5','2','5','3','5','4','5','5','5','6','5','7','5','8','5','9',
'6','0','6','1','6','2','6','3','6','4','6','5','6','6','6','7','6','8','6','9',
'7','0','7','1','7','2','7','3','7','4','7','5','7','6','7','7','7','8','7','9',
'8','0','8','1','8','2','8','3','8','4','8','5','8','6','8','7','8','8','8','9',
'9','0','9','1','9','2','9','3','9','4','9','5','9','6','9','7','9','8','9','9'
};
static inline int u64toa(char* buf, uint64_t value) {
const char* b = buf;
if (value < 100000000) {
uint32_t v = (uint32_t)(value);
if (v < 10000) {
const uint32_t d1 = (v / 100) << 1;
const uint32_t d2 = (v % 100) << 1;
if (v >= 1000)
*buf++ = gDigitsLut[d1];
if (v >= 100)
*buf++ = gDigitsLut[d1 + 1];
if (v >= 10)
*buf++ = gDigitsLut[d2];
*buf++ = gDigitsLut[d2 + 1];
}
else {
// value = bbbbcccc
const uint32_t b = v / 10000;
const uint32_t c = v % 10000;
const uint32_t d1 = (b / 100) << 1;
const uint32_t d2 = (b % 100) << 1;
const uint32_t d3 = (c / 100) << 1;
const uint32_t d4 = (c % 100) << 1;
if (value >= 10000000)
*buf++ = gDigitsLut[d1];
if (value >= 1000000)
*buf++ = gDigitsLut[d1 + 1];
if (value >= 100000)
*buf++ = gDigitsLut[d2];
*buf++ = gDigitsLut[d2 + 1];
*buf++ = gDigitsLut[d3];
*buf++ = gDigitsLut[d3 + 1];
*buf++ = gDigitsLut[d4];
*buf++ = gDigitsLut[d4 + 1];
}
}
// *buf = '\0';
return buf - b;
}

116
thirdparty/picohttpparser/src/README.md vendored 100644
View File

@ -0,0 +1,116 @@
PicoHTTPParser
=============
Copyright (c) 2009-2014 [Kazuho Oku](https://github.com/kazuho), [Tokuhiro Matsuno](https://github.com/tokuhirom), [Daisuke Murase](https://github.com/typester), [Shigeo Mitsunari](https://github.com/herumi)
PicoHTTPParser is a tiny, primitive, fast HTTP request/response parser.
Unlike most parsers, it is stateless and does not allocate memory by itself.
All it does is accept pointer to buffer and the output structure, and setups the pointers in the latter to point at the necessary portions of the buffer.
The code is widely deployed within Perl applications through popular modules that use it, including [Plack](https://metacpan.org/pod/Plack), [Starman](https://metacpan.org/pod/Starman), [Starlet](https://metacpan.org/pod/Starlet), [Furl](https://metacpan.org/pod/Furl). It is also the HTTP/1 parser of [H2O](https://github.com/h2o/h2o).
Check out [test.c] to find out how to use the parser.
The software is dual-licensed under the Perl License or the MIT License.
Usage
-----
The library exposes four functions: `phr_parse_request`, `phr_parse_response`, `phr_parse_headers`, `phr_decode_chunked`.
### phr_parse_request
The example below reads an HTTP request from socket `sock` using `read(2)`, parses it using `phr_parse_request`, and prints the details.
```c
char buf[4096], *method, *path;
int pret, minor_version;
struct phr_header headers[100];
size_t buflen = 0, prevbuflen = 0, method_len, path_len, num_headers;
ssize_t rret;
while (1) {
/* read the request */
while ((rret = read(sock, buf + buflen, sizeof(buf) - buflen)) == -1 && errno == EINTR)
;
if (rret <= 0)
return IOError;
prevbuflen = buflen;
buflen += rret;
/* parse the request */
num_headers = sizeof(headers) / sizeof(headers[0]);
pret = phr_parse_request(buf, buflen, &method, &method_len, &path, &path_len,
&minor_version, headers, &num_headers, prevbuflen);
if (pret > 0)
break; /* successfully parsed the request */
else if (pret == -1)
return ParseError;
/* request is incomplete, continue the loop */
assert(pret == -2);
if (buflen == sizeof(buf))
return RequestIsTooLongError;
}
printf("request is %d bytes long\n", pret);
printf("method is %.*s\n", (int)method_len, method);
printf("path is %.*s\n", (int)path_len, path);
printf("HTTP version is 1.%d\n", minor_version);
printf("headers:\n");
for (i = 0; i != num_headers; ++i) {
printf("%.*s: %.*s\n", (int)headers[i].name_len, headers[i].name,
(int)headers[i].value_len, headers[i].value);
}
```
### phr_parse_response, phr_parse_headers
`phr_parse_response` and `phr_parse_headers` provide similar interfaces as `phr_parse_request`. `phr_parse_response` parses an HTTP response, and `phr_parse_headers` parses the headers only.
### phr_decode_chunked
The example below decodes incoming data in chunked-encoding. The data is decoded in-place.
```c
struct phr_chunked_decoder decoder = {}; /* zero-clear */
char *buf = malloc(4096);
size_t size = 0, capacity = 4096, rsize;
ssize_t rret, pret;
/* set consume_trailer to 1 to discard the trailing header, or the application
* should call phr_parse_headers to parse the trailing header */
decoder.consume_trailer = 1;
do {
/* expand the buffer if necessary */
if (size == capacity) {
capacity *= 2;
buf = realloc(buf, capacity);
assert(buf != NULL);
}
/* read */
while ((rret = read(sock, buf + size, capacity - size)) == -1 && errno == EINTR)
;
if (rret <= 0)
return IOError;
/* decode */
rsize = rret;
pret = phr_decode_chunked(&decoder, buf + size, &rsize);
if (pret == -1)
return ParseError;
size += rsize;
} while (pret == -2);
/* successfully decoded the chunked data */
assert(pret >= 0);
printf("decoded data is at %p (%zu bytes)\n", buf, size);
```
Benchmark
---------
![benchmark results](http://i.gyazo.com/a85c18d3162dfb46b485bb41e0ad443a.png)
The benchmark code is from [fukamachi/fast-http@6b91103](https://github.com/fukamachi/fast-http/tree/6b9110347c7a3407310c08979aefd65078518478).
The internals of picohttpparser has been described to some extent in [my blog entry]( http://blog.kazuhooku.com/2014/11/the-internals-h2o-or-how-to-write-fast.html).

645
thirdparty/picohttpparser/src/picohttpparser.c vendored 100644
View File

@ -0,0 +1,645 @@
/*
* Copyright (c) 2009-2014 Kazuho Oku, Tokuhiro Matsuno, Daisuke Murase,
* Shigeo Mitsunari
*
* The software is licensed under either the MIT License (below) or the Perl
* license.
*
* Permission is hereby granted, free of charge, to any person obtaining a copy
* of this software and associated documentation files (the "Software"), to
* deal in the Software without restriction, including without limitation the
* rights to use, copy, modify, merge, publish, distribute, sublicense, and/or
* sell copies of the Software, and to permit persons to whom the Software is
* furnished to do so, subject to the following conditions:
*
* The above copyright notice and this permission notice shall be included in
* all copies or substantial portions of the Software.
*
* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
* IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
* FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
* AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
* LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
* FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS
* IN THE SOFTWARE.
*/
#include <assert.h>
#include <stddef.h>
#include <string.h>
#ifdef __SSE4_2__
#ifdef _MSC_VER
#include <nmmintrin.h>
#else
#include <x86intrin.h>
#endif
#endif
#include "picohttpparser.h"
#if __GNUC__ >= 3
#define likely(x) __builtin_expect(!!(x), 1)
#define unlikely(x) __builtin_expect(!!(x), 0)
#else
#define likely(x) (x)
#define unlikely(x) (x)
#endif
#ifdef _MSC_VER
#define ALIGNED(n) _declspec(align(n))
#else
#define ALIGNED(n) __attribute__((aligned(n)))
#endif
#define IS_PRINTABLE_ASCII(c) ((unsigned char)(c)-040u < 0137u)
#define CHECK_EOF() \
if (buf == buf_end) { \
*ret = -2; \
return NULL; \
}
#define EXPECT_CHAR_NO_CHECK(ch) \
if (*buf++ != ch) { \
*ret = -1; \
return NULL; \
}
#define EXPECT_CHAR(ch) \
CHECK_EOF(); \
EXPECT_CHAR_NO_CHECK(ch);
#define ADVANCE_TOKEN(tok, toklen) \
do { \
const char *tok_start = buf; \
static const char ALIGNED(16) ranges2[16] = "\000\040\177\177"; \
int found2; \
buf = findchar_fast(buf, buf_end, ranges2, 4, &found2); \
if (!found2) { \
CHECK_EOF(); \
} \
while (1) { \
if (*buf == ' ') { \
break; \
} else if (unlikely(!IS_PRINTABLE_ASCII(*buf))) { \
if ((unsigned char)*buf < '\040' || *buf == '\177') { \
*ret = -1; \
return NULL; \
} \
} \
++buf; \
CHECK_EOF(); \
} \
tok = tok_start; \
toklen = buf - tok_start; \
} while (0)
static const char *token_char_map = "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"
"\0\1\0\1\1\1\1\1\0\0\1\1\0\1\1\0\1\1\1\1\1\1\1\1\1\1\0\0\0\0\0\0"
"\0\1\1\1\1\1\1\1\1\1\1\1\1\1\1\1\1\1\1\1\1\1\1\1\1\1\1\0\0\0\1\1"
"\1\1\1\1\1\1\1\1\1\1\1\1\1\1\1\1\1\1\1\1\1\1\1\1\1\1\1\0\1\0\1\0"
"\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"
"\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"
"\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"
"\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0";
static const char *findchar_fast(const char *buf, const char *buf_end, const char *ranges, size_t ranges_size, int *found)
{
*found = 0;
#if __SSE4_2__
if (likely(buf_end - buf >= 16)) {
__m128i ranges16 = _mm_loadu_si128((const __m128i *)ranges);
size_t left = (buf_end - buf) & ~15;
do {
__m128i b16 = _mm_loadu_si128((const __m128i *)buf);
int r = _mm_cmpestri(ranges16, ranges_size, b16, 16, _SIDD_LEAST_SIGNIFICANT | _SIDD_CMP_RANGES | _SIDD_UBYTE_OPS);
if (unlikely(r != 16)) {
buf += r;
*found = 1;
break;
}
buf += 16;
left -= 16;
} while (likely(left != 0));
}
#else
/* suppress unused parameter warning */
(void)buf_end;
(void)ranges;
(void)ranges_size;
#endif
return buf;
}
static const char *get_token_to_eol(const char *buf, const char *buf_end, const char **token, size_t *token_len, int *ret)
{
const char *token_start = buf;
#ifdef __SSE4_2__
static const char ALIGNED(16) ranges1[16] = "\0\010" /* allow HT */
"\012\037" /* allow SP and up to but not including DEL */
"\177\177"; /* allow chars w. MSB set */
int found;
buf = findchar_fast(buf, buf_end, ranges1, 6, &found);
if (found)
goto FOUND_CTL;
#else
/* find non-printable char within the next 8 bytes, this is the hottest code; manually inlined */
while (likely(buf_end - buf >= 8)) {
#define DOIT() \
do { \
if (unlikely(!IS_PRINTABLE_ASCII(*buf))) \
goto NonPrintable; \
++buf; \
} while (0)
DOIT();
DOIT();
DOIT();
DOIT();
DOIT();
DOIT();
DOIT();
DOIT();
#undef DOIT
continue;
NonPrintable:
if ((likely((unsigned char)*buf < '\040') && likely(*buf != '\011')) || unlikely(*buf == '\177')) {
goto FOUND_CTL;
}
++buf;
}
#endif
for (;; ++buf) {
CHECK_EOF();
if (unlikely(!IS_PRINTABLE_ASCII(*buf))) {
if ((likely((unsigned char)*buf < '\040') && likely(*buf != '\011')) || unlikely(*buf == '\177')) {
goto FOUND_CTL;
}
}
}
FOUND_CTL:
if (likely(*buf == '\015')) {
++buf;
EXPECT_CHAR('\012');
*token_len = buf - 2 - token_start;
} else if (*buf == '\012') {
*token_len = buf - token_start;
++buf;
} else {
*ret = -1;
return NULL;
}
*token = token_start;
return buf;
}
static const char *is_complete(const char *buf, const char *buf_end, size_t last_len, int *ret)
{
int ret_cnt = 0;
buf = last_len < 3 ? buf : buf + last_len - 3;
while (1) {
CHECK_EOF();
if (*buf == '\015') {
++buf;
CHECK_EOF();
EXPECT_CHAR('\012');
++ret_cnt;
} else if (*buf == '\012') {
++buf;
++ret_cnt;
} else {
++buf;
ret_cnt = 0;
}
if (ret_cnt == 2) {
return buf;
}
}
*ret = -2;
return NULL;
}
#define PARSE_INT(valp_, mul_) \
if (*buf < '0' || '9' < *buf) { \
buf++; \
*ret = -1; \
return NULL; \
} \
*(valp_) = (mul_) * (*buf++ - '0');
#define PARSE_INT_3(valp_) \
do { \
int res_ = 0; \
PARSE_INT(&res_, 100) \
*valp_ = res_; \
PARSE_INT(&res_, 10) \
*valp_ += res_; \
PARSE_INT(&res_, 1) \
*valp_ += res_; \
} while (0)
/* returned pointer is always within [buf, buf_end), or null */
static const char *parse_http_version(const char *buf, const char *buf_end, int *minor_version, int *ret)
{
/* we want at least [HTTP/1.<two chars>] to try to parse */
if (buf_end - buf < 9) {
*ret = -2;
return NULL;
}
EXPECT_CHAR_NO_CHECK('H');
EXPECT_CHAR_NO_CHECK('T');
EXPECT_CHAR_NO_CHECK('T');
EXPECT_CHAR_NO_CHECK('P');
EXPECT_CHAR_NO_CHECK('/');
EXPECT_CHAR_NO_CHECK('1');
EXPECT_CHAR_NO_CHECK('.');
PARSE_INT(minor_version, 1);
return buf;
}
static const char *parse_headers(const char *buf, const char *buf_end, struct phr_header *headers, size_t *num_headers,
size_t max_headers, int *ret)
{
for (;; ++*num_headers) {
CHECK_EOF();
if (*buf == '\015') {
++buf;
EXPECT_CHAR('\012');
break;
} else if (*buf == '\012') {
++buf;
break;
}
if (*num_headers == max_headers) {
*ret = -1;
return NULL;
}
if (!(*num_headers != 0 && (*buf == ' ' || *buf == '\t'))) {
/* parsing name, but do not discard SP before colon, see
* http://www.mozilla.org/security/announce/2006/mfsa2006-33.html */
headers[*num_headers].name = buf;
static const char ALIGNED(16) ranges1[] = "\x00 " /* control chars and up to SP */
"\"\"" /* 0x22 */
"()" /* 0x28,0x29 */
",," /* 0x2c */
"//" /* 0x2f */
":@" /* 0x3a-0x40 */
"[]" /* 0x5b-0x5d */
"{\377"; /* 0x7b-0xff */
int found;
buf = findchar_fast(buf, buf_end, ranges1, sizeof(ranges1) - 1, &found);
if (!found) {
CHECK_EOF();
}
while (1) {
if (*buf == ':') {
break;
} else if (!token_char_map[(unsigned char)*buf]) {
*ret = -1;
return NULL;
}
++buf;
CHECK_EOF();
}
if ((headers[*num_headers].name_len = buf - headers[*num_headers].name) == 0) {
*ret = -1;
return NULL;
}
++buf;
for (;; ++buf) {
CHECK_EOF();
if (!(*buf == ' ' || *buf == '\t')) {
break;
}
}
} else {
headers[*num_headers].name = NULL;
headers[*num_headers].name_len = 0;
}
const char *value;
size_t value_len;
if ((buf = get_token_to_eol(buf, buf_end, &value, &value_len, ret)) == NULL) {
return NULL;
}
/* remove trailing SPs and HTABs */
const char *value_end = value + value_len;
for (; value_end != value; --value_end) {
const char c = *(value_end - 1);
if (!(c == ' ' || c == '\t')) {
break;
}
}
headers[*num_headers].value = value;
headers[*num_headers].value_len = value_end - value;
}
return buf;
}
static const char *parse_request(const char *buf, const char *buf_end, const char **method, size_t *method_len, const char **path,
size_t *path_len, int *minor_version, struct phr_header *headers, size_t *num_headers,
size_t max_headers, int *ret)
{
/* skip first empty line (some clients add CRLF after POST content) */
CHECK_EOF();
if (*buf == '\015') {
++buf;
EXPECT_CHAR('\012');
} else if (*buf == '\012') {
++buf;
}
/* parse request line */
ADVANCE_TOKEN(*method, *method_len);
do {
++buf;
} while (*buf == ' ');
ADVANCE_TOKEN(*path, *path_len);
do {
++buf;
} while (*buf == ' ');
if (*method_len == 0 || *path_len == 0) {
*ret = -1;
return NULL;
}
if ((buf = parse_http_version(buf, buf_end, minor_version, ret)) == NULL) {
return NULL;
}
if (*buf == '\015') {
++buf;
EXPECT_CHAR('\012');
} else if (*buf == '\012') {
++buf;
} else {
*ret = -1;
return NULL;
}
return parse_headers(buf, buf_end, headers, num_headers, max_headers, ret);
}
int phr_parse_request(const char *buf_start, size_t len, const char **method, size_t *method_len, const char **path,
size_t *path_len, int *minor_version, struct phr_header *headers, size_t *num_headers, size_t last_len)
{
const char *buf = buf_start, *buf_end = buf_start + len;
size_t max_headers = *num_headers;
int r;
*method = NULL;
*method_len = 0;
*path = NULL;
*path_len = 0;
*minor_version = -1;
*num_headers = 0;
/* if last_len != 0, check if the request is complete (a fast countermeasure
againt slowloris */
if (last_len != 0 && is_complete(buf, buf_end, last_len, &r) == NULL) {
return r;
}
if ((buf = parse_request(buf, buf_end, method, method_len, path, path_len, minor_version, headers, num_headers, max_headers,
&r)) == NULL) {
return r;
}
return (int)(buf - buf_start);
}
static const char *parse_response(const char *buf, const char *buf_end, int *minor_version, int *status, const char **msg,
size_t *msg_len, struct phr_header *headers, size_t *num_headers, size_t max_headers, int *ret)
{
/* parse "HTTP/1.x" */
if ((buf = parse_http_version(buf, buf_end, minor_version, ret)) == NULL) {
return NULL;
}
/* skip space */
if (*buf != ' ') {
*ret = -1;
return NULL;
}
do {
++buf;
} while (*buf == ' ');
/* parse status code, we want at least [:digit:][:digit:][:digit:]<other char> to try to parse */
if (buf_end - buf < 4) {
*ret = -2;
return NULL;
}
PARSE_INT_3(status);
/* get message includig preceding space */
if ((buf = get_token_to_eol(buf, buf_end, msg, msg_len, ret)) == NULL) {
return NULL;
}
if (*msg_len == 0) {
/* ok */
} else if (**msg == ' ') {
/* remove preceding space */
do {
++*msg;
--*msg_len;
} while (**msg == ' ');
} else {
/* garbage found after status code */
*ret = -1;
return NULL;
}
return parse_headers(buf, buf_end, headers, num_headers, max_headers, ret);
}
int phr_parse_response(const char *buf_start, size_t len, int *minor_version, int *status, const char **msg, size_t *msg_len,
struct phr_header *headers, size_t *num_headers, size_t last_len)
{
const char *buf = buf_start, *buf_end = buf + len;
size_t max_headers = *num_headers;
int r;
*minor_version = -1;
*status = 0;
*msg = NULL;
*msg_len = 0;
*num_headers = 0;
/* if last_len != 0, check if the response is complete (a fast countermeasure
against slowloris */
if (last_len != 0 && is_complete(buf, buf_end, last_len, &r) == NULL) {
return r;
}
if ((buf = parse_response(buf, buf_end, minor_version, status, msg, msg_len, headers, num_headers, max_headers, &r)) == NULL) {
return r;
}
return (int)(buf - buf_start);
}
int phr_parse_headers(const char *buf_start, size_t len, struct phr_header *headers, size_t *num_headers, size_t last_len)
{
const char *buf = buf_start, *buf_end = buf + len;
size_t max_headers = *num_headers;
int r;
*num_headers = 0;
/* if last_len != 0, check if the response is complete (a fast countermeasure
against slowloris */
if (last_len != 0 && is_complete(buf, buf_end, last_len, &r) == NULL) {
return r;
}
if ((buf = parse_headers(buf, buf_end, headers, num_headers, max_headers, &r)) == NULL) {
return r;
}
return (int)(buf - buf_start);
}
enum {
CHUNKED_IN_CHUNK_SIZE,
CHUNKED_IN_CHUNK_EXT,
CHUNKED_IN_CHUNK_DATA,
CHUNKED_IN_CHUNK_CRLF,
CHUNKED_IN_TRAILERS_LINE_HEAD,
CHUNKED_IN_TRAILERS_LINE_MIDDLE
};
static int decode_hex(int ch)
{
if ('0' <= ch && ch <= '9') {
return ch - '0';
} else if ('A' <= ch && ch <= 'F') {
return ch - 'A' + 0xa;
} else if ('a' <= ch && ch <= 'f') {
return ch - 'a' + 0xa;
} else {
return -1;
}
}
ssize_t phr_decode_chunked(struct phr_chunked_decoder *decoder, char *buf, size_t *_bufsz)
{
size_t dst = 0, src = 0, bufsz = *_bufsz;
ssize_t ret = -2; /* incomplete */
while (1) {
switch (decoder->_state) {
case CHUNKED_IN_CHUNK_SIZE:
for (;; ++src) {
int v;
if (src == bufsz)
goto Exit;
if ((v = decode_hex(buf[src])) == -1) {
if (decoder->_hex_count == 0) {
ret = -1;
goto Exit;
}
break;
}
if (decoder->_hex_count == sizeof(size_t) * 2) {
ret = -1;
goto Exit;
}
decoder->bytes_left_in_chunk = decoder->bytes_left_in_chunk * 16 + v;
++decoder->_hex_count;
}
decoder->_hex_count = 0;
decoder->_state = CHUNKED_IN_CHUNK_EXT;
/* fallthru */
case CHUNKED_IN_CHUNK_EXT:
/* RFC 7230 A.2 "Line folding in chunk extensions is disallowed" */
for (;; ++src) {
if (src == bufsz)
goto Exit;
if (buf[src] == '\012')
break;
}
++src;
if (decoder->bytes_left_in_chunk == 0) {
if (decoder->consume_trailer) {
decoder->_state = CHUNKED_IN_TRAILERS_LINE_HEAD;
break;
} else {
goto Complete;
}
}
decoder->_state = CHUNKED_IN_CHUNK_DATA;
/* fallthru */
case CHUNKED_IN_CHUNK_DATA: {
size_t avail = bufsz - src;
if (avail < decoder->bytes_left_in_chunk) {
if (dst != src)
memmove(buf + dst, buf + src, avail);
src += avail;
dst += avail;
decoder->bytes_left_in_chunk -= avail;
goto Exit;
}
if (dst != src)
memmove(buf + dst, buf + src, decoder->bytes_left_in_chunk);
src += decoder->bytes_left_in_chunk;
dst += decoder->bytes_left_in_chunk;
decoder->bytes_left_in_chunk = 0;
decoder->_state = CHUNKED_IN_CHUNK_CRLF;
}
/* fallthru */
case CHUNKED_IN_CHUNK_CRLF:
for (;; ++src) {
if (src == bufsz)
goto Exit;
if (buf[src] != '\015')
break;
}
if (buf[src] != '\012') {
ret = -1;
goto Exit;
}
++src;
decoder->_state = CHUNKED_IN_CHUNK_SIZE;
break;
case CHUNKED_IN_TRAILERS_LINE_HEAD:
for (;; ++src) {
if (src == bufsz)
goto Exit;
if (buf[src] != '\015')
break;
}
if (buf[src++] == '\012')
goto Complete;
decoder->_state = CHUNKED_IN_TRAILERS_LINE_MIDDLE;
/* fallthru */
case CHUNKED_IN_TRAILERS_LINE_MIDDLE:
for (;; ++src) {
if (src == bufsz)
goto Exit;
if (buf[src] == '\012')
break;
}
++src;
decoder->_state = CHUNKED_IN_TRAILERS_LINE_HEAD;
break;
default:
assert(!"decoder is corrupt");
}
}
Complete:
ret = bufsz - src;
Exit:
if (dst != src)
memmove(buf + dst, buf + src, bufsz - src);
*_bufsz = dst;
return ret;
}
int phr_decode_chunked_is_in_data(struct phr_chunked_decoder *decoder)
{
return decoder->_state == CHUNKED_IN_CHUNK_DATA;
}
#undef CHECK_EOF
#undef EXPECT_CHAR
#undef ADVANCE_TOKEN

87
thirdparty/picohttpparser/src/picohttpparser.h vendored 100644
View File

@ -0,0 +1,87 @@
/*
* Copyright (c) 2009-2014 Kazuho Oku, Tokuhiro Matsuno, Daisuke Murase,
* Shigeo Mitsunari
*
* The software is licensed under either the MIT License (below) or the Perl
* license.
*
* Permission is hereby granted, free of charge, to any person obtaining a copy
* of this software and associated documentation files (the "Software"), to
* deal in the Software without restriction, including without limitation the
* rights to use, copy, modify, merge, publish, distribute, sublicense, and/or
* sell copies of the Software, and to permit persons to whom the Software is
* furnished to do so, subject to the following conditions:
*
* The above copyright notice and this permission notice shall be included in
* all copies or substantial portions of the Software.
*
* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
* IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
* FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
* AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
* LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
* FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS
* IN THE SOFTWARE.
*/
#ifndef picohttpparser_h
#define picohttpparser_h
#include <sys/types.h>
#ifdef _MSC_VER
#define ssize_t intptr_t
#endif
#ifdef __cplusplus
extern "C" {
#endif
/* contains name and value of a header (name == NULL if is a continuing line
* of a multiline header */
struct phr_header {
const char *name;
size_t name_len;
const char *value;
size_t value_len;
};
/* returns number of bytes consumed if successful, -2 if request is partial,
* -1 if failed */
int phr_parse_request(const char *buf, size_t len, const char **method, size_t *method_len, const char **path, size_t *path_len,
int *minor_version, struct phr_header *headers, size_t *num_headers, size_t last_len);
/* ditto */
int phr_parse_response(const char *_buf, size_t len, int *minor_version, int *status, const char **msg, size_t *msg_len,
struct phr_header *headers, size_t *num_headers, size_t last_len);
/* ditto */
int phr_parse_headers(const char *buf, size_t len, struct phr_header *headers, size_t *num_headers, size_t last_len);
/* should be zero-filled before start */
struct phr_chunked_decoder {
size_t bytes_left_in_chunk; /* number of bytes left in current chunk */
char consume_trailer; /* if trailing headers should be consumed */
char _hex_count;
char _state;
};
/* the function rewrites the buffer given as (buf, bufsz) removing the chunked-
* encoding headers. When the function returns without an error, bufsz is
* updated to the length of the decoded data available. Applications should
* repeatedly call the function while it returns -2 (incomplete) every time
* supplying newly arrived data. If the end of the chunked-encoded data is
* found, the function returns a non-negative number indicating the number of
* octets left undecoded at the tail of the supplied buffer. Returns -1 on
* error.
*/
ssize_t phr_decode_chunked(struct phr_chunked_decoder *decoder, char *buf, size_t *bufsz);
/* returns if the chunked decoder is in middle of chunked data */
int phr_decode_chunked_is_in_data(struct phr_chunked_decoder *decoder);
#ifdef __cplusplus
}
#endif
#endif

234
vlib/picoev/picoev.v 100644
View File

@ -0,0 +1,234 @@
// Copyright (c) 2019 Alexander Medvednikov. All rights reserved.
// Use of this source code is governed by an MIT license
// that can be found in the LICENSE file.
module picoev
import picohttpparser
#include <errno.h>
#include <sys/socket.h>
#include <netinet/in.h>
#include <netinet/tcp.h>
#include <fcntl.h>
#include <signal.h>
#flag -I @VROOT/thirdparty/picoev
#flag -L @VROOT/thirdparty/picoev
#flag @VROOT/thirdparty/picoev/picoev.o
#include "src/picoev.h"
const (
MAX_FDS = 1024
TIMEOUT_SECS = 8
MAX_TIMEOUT = 10
MAX_READ = 4096
MAX_WRITE = 8192
)
struct C.in_addr {
mut:
s_addr int
}
struct C.sockaddr_in {
mut:
sin_family int
sin_port int
sin_addr C.in_addr
}
struct C.sockaddr_storage {}
fn C.socket() int
fn C.setsockopt() int
fn C.htonl() int
fn C.htons() int
fn C.bind() int
fn C.listen() int
fn C.accept() int
fn C.getaddrinfo() int
fn C.connect() int
fn C.send() int
fn C.recv() int
fn C.read() int
fn C.shutdown() int
fn C.close() int
fn C.ntohs() int
fn C.getsockname() int
fn C.fcntl() int
fn C.write() int
struct C.picoev_loop {}
struct Picoev {
loop *C.picoev_loop
cb fn(req picohttpparser.Request, res mut picohttpparser.Response)
mut:
date byteptr
buf byteptr
idx [1024]int
out byteptr
oidx [1024]int
}
fn picoev_del(*C.picoev_loop, int) int
fn picoev_set_timeout(*C.picoev_loop, int, int)
fn picoev_add(*C.picoev_loop, int, int, int, *C.picoev_handler, voidptr) int
fn picoev_init(int) int
fn picoev_create_loop(int) *C.picoev_loop
fn picoev_loop_once(*C.picoev_loop, int) int
fn picoev_destroy_loop(*C.picoev_loop) int
fn picoev_deinit() int
[inline]
fn setup_sock(fd int) {
on := 1
if C.setsockopt(fd, C.IPPROTO_TCP, C.TCP_NODELAY, &on, sizeof(int)) < 0 {
println('setup_sock.setup_sock failed')
}
if C.fcntl(fd, C.F_SETFL, C.O_NONBLOCK) != 0 {
println('fcntl failed')
}
}
[inline]
fn close_conn(loop *C.picoev_loop, fd int) {
C.picoev_del(loop, fd)
C.close(fd)
}
[inline]
fn myread(fd int, b byteptr, max_len, idx int) int {
return C.read(fd, b + idx, max_len - idx)
}
[inline]
fn mysubstr(s byteptr, from, len int) string {
return tos(s + from, len)
}
fn rw_callback(loop *C.picoev_loop, fd, events int, cb_arg voidptr) {
mut p := *Picoev(cb_arg)
if (events & C.PICOEV_TIMEOUT) != 0 {
close_conn(loop, fd)
p.idx[fd] = 0
return
}
else if (events & C.PICOEV_READ) != 0 {
C.picoev_set_timeout(loop, fd, TIMEOUT_SECS)
buf := (p.buf + fd * MAX_READ)
idx := p.idx[fd]
mut r := myread(fd, buf, MAX_READ, idx)
if (r == 0) {
close_conn(loop, fd)
p.idx[fd] = 0
return
} else if (r == -1) {
if errno == C.EAGAIN || errno == C.EWOULDBLOCK {
//
} else {
close_conn(loop, fd)
p.idx[fd] = 0
return
}
} else {
r += idx
mut s := tos(buf, r)
out := (p.out + fd * MAX_WRITE)
mut res := picohttpparser.Response{
fd: fd
date: p.date
buf_start: out
buf: out + p.oidx[fd]
}
mut req := picohttpparser.Request{}
for {
pret := req.parse_request_path_pipeline(s)
if pret <= 0 && s.len > 0 {
C.memmove(buf, s.str, s.len)
p.idx[fd] = s.len
p.oidx[fd] = int(res.buf - res.buf_start)
break
}
p.cb(req, mut res)
if pret >= s.len {
p.idx[fd] = 0
p.oidx[fd] = 0
if res.end() < 0 {
close_conn(loop, fd)
return
}
break
}
s = mysubstr(buf, pret, s.len - pret)
}
}
}
}
fn accept_callback(loop *C.picoev_loop, fd, events int, cb_arg voidptr) {
newfd := C.accept(fd, 0, 0)
if newfd != -1 {
setup_sock(newfd)
C.picoev_add(loop, newfd, C.PICOEV_READ, TIMEOUT_SECS, rw_callback, cb_arg)
}
}
pub fn new(port int, cb voidptr) &Picoev {
fd := C.socket(C.AF_INET, C.SOCK_STREAM, 0)
assert fd != -1
flag := 1
assert C.setsockopt(fd, C.SOL_SOCKET, C.SO_REUSEADDR, &flag, sizeof(int)) == 0
assert C.setsockopt(fd, C.SOL_SOCKET, C.SO_REUSEPORT, &flag, sizeof(int)) == 0
$if linux {
assert C.setsockopt(fd, C.IPPROTO_TCP, C.TCP_QUICKACK, &flag, sizeof(int)) == 0
timeout := 10
assert C.setsockopt(fd, C.IPPROTO_TCP, C.TCP_DEFER_ACCEPT, &timeout, sizeof(int)) == 0
queue_len := 4096
assert C.setsockopt(fd, C.IPPROTO_TCP, C.TCP_FASTOPEN, &queue_len, sizeof(int)) == 0
}
mut addr := C.sockaddr_in{}
addr.sin_family = C.AF_INET
addr.sin_port = C.htons(port)
addr.sin_addr.s_addr = C.htonl(C.INADDR_ANY)
size := 16 // sizeof(C.sockaddr_in)
bind_res := C.bind(fd, &addr, size)
assert bind_res == 0
listen_res := C.listen(fd, C.SOMAXCONN)
assert listen_res == 0
setup_sock(fd)
C.picoev_init(MAX_FDS)
loop := C.picoev_create_loop(MAX_TIMEOUT)
pv := &Picoev{
loop: loop
cb: cb
date: C.get_date()
buf: malloc(MAX_FDS * MAX_READ + 1)
out: malloc(MAX_FDS * MAX_WRITE + 1)
}
C.picoev_add(loop, fd, C.PICOEV_READ, 0, accept_callback, pv)
go update_date(pv)
return pv
}
pub fn (p Picoev) serve() {
for {
C.picoev_loop_once(p.loop, 1)
}
}
fn update_date(p mut Picoev) {
for {
p.date = C.get_date()
C.usleep(1000000)
}
}

24
vlib/picohttpparser/misc.v 100644
View File

@ -0,0 +1,24 @@
module picohttpparser
[inline]
fn cpy_str(dst byteptr, src string) int {
C.memcpy(dst, src.str, src.len)
return src.len
}
[inline]
fn cpy(dst, src byteptr, len int) int {
C.memcpy(dst, src, len)
return len
}
[inline]
pub fn cmp(dst, src string) bool {
if dst.len != src.len { return false }
return C.memcmp(dst.str, src.str, src.len) == 0
}
[inline]
pub fn cmpn(dst, src string, n int) bool {
return C.memcmp(dst.str, src.str, n) == 0
}

29
vlib/picohttpparser/picohttpparser.v 100644
View File

@ -0,0 +1,29 @@
// Copyright (c) 2019 Alexander Medvednikov. All rights reserved.
// Use of this source code is governed by an MIT license
// that can be found in the LICENSE file.
module picohttpparser
#flag -I @VROOT/thirdparty/picohttpparser
#flag -L @VROOT/thirdparty/picohttpparser
#flag @VROOT/thirdparty/picohttpparser/picohttpparser.o
#include "picohttpparser.h"
struct C.phr_header {
pub:
name charptr
name_len int
value charptr
value_len int
}
struct C.phr_header_t {}
fn phr_parse_request() int
fn phr_parse_response() int
fn phr_parse_headers() int
fn phr_parse_request_path() int
fn phr_parse_request_path_pipeline() int
fn C.get_date() byteptr
fn C.u64toa() int
fn C.memcmp() int

67
vlib/picohttpparser/request.v 100644
View File

@ -0,0 +1,67 @@
module picohttpparser
pub struct Request {
mut:
method string
path string
headers *C.phr_header_t
num_headers u64
}
[inline]
pub fn (r mut Request) parse_request(s string, headers *C.phr_header_t, max_headers int) int {
method_len := u64(0)
path_len := u64(0)
minor_version := 0
num_headers := u64(max_headers)
pret := C.phr_parse_request(
s.str, s.len,
&r.method, &method_len,
&r.path, &path_len,
&minor_version,
headers, &num_headers,
0
)
if pret > 0 {
r.method = tos(r.method.str, int(method_len))
r.path = tos(r.path.str, int(path_len))
r.headers = headers
r.num_headers = num_headers
}
return pret
}
[inline]
pub fn (r mut Request) parse_request_path(s string) int {
method_len := u64(0)
path_len := u64(0)
pret := C.phr_parse_request_path(
s.str, s.len,
&r.method, &method_len,
&r.path, &path_len
)
if pret > 0 {
r.method = tos(r.method.str, int(method_len))
r.path = tos(r.path.str, int(path_len))
}
return pret
}
[inline]
pub fn (r mut Request) parse_request_path_pipeline(s string) int {
method_len := u64(0)
path_len := u64(0)
pret := C.phr_parse_request_path_pipeline(
s.str, s.len,
&r.method, &method_len,
&r.path, &path_len
)
if pret > 0 {
r.method = tos(r.method.str, int(method_len))
r.path = tos(r.path.str, int(path_len))
}
return pret
}

96
vlib/picohttpparser/response.v 100644
View File

@ -0,0 +1,96 @@
module picohttpparser
pub struct Response {
fd int
pub:
date byteptr
buf_start byteptr
mut:
buf byteptr
}
[inline]
pub fn (r mut Response) http_ok() &Response {
r.buf += cpy_str(r.buf, "HTTP/1.1 200 OK\r\n")
return r
}
[inline]
pub fn (r mut Response) header(k, v string) &Response {
r.buf += cpy_str(r.buf, k)
r.buf += cpy_str(r.buf, ": ")
r.buf += cpy_str(r.buf, v)
r.buf += cpy_str(r.buf, "\r\n")
return r
}
[inline]
pub fn (r mut Response) header_date() &Response {
r.buf += cpy_str(r.buf, "Date: ")
r.buf += cpy(r.buf, r.date, 29)
r.buf += cpy_str(r.buf, "\r\n")
return r
}
[inline]
pub fn (r mut Response) header_server() &Response {
r.buf += cpy_str(r.buf, "Server: V\r\n")
return r
}
[inline]
pub fn (r mut Response) content_type(s string) &Response {
r.buf += cpy_str(r.buf, "Content-Type: ")
r.buf += cpy_str(r.buf, s)
r.buf += cpy_str(r.buf, "\r\n")
return r
}
[inline]
pub fn (r mut Response) plain() &Response {
r.buf += cpy_str(r.buf, "Content-Type: text/plain\r\n")
return r
}
[inline]
pub fn (r mut Response) json() &Response {
r.buf += cpy_str(r.buf, "Content-Type: application/json\r\n")
return r
}
[inline]
pub fn (r mut Response) body(body string) {
r.buf += cpy_str(r.buf, "Content-Length: ")
r.buf += C.u64toa(r.buf, body.len)
r.buf += cpy_str(r.buf, "\r\n\r\n")
r.buf += cpy_str(r.buf, body)
}
[inline]
pub fn (r mut Response) http_404() {
r.buf += cpy_str(r.buf, 'HTTP/1.1 404 Not Found\r\nContent-Length: 0\r\n\r\n')
}
[inline]
pub fn (r mut Response) http_405() {
r.buf += cpy_str(r.buf, 'HTTP/1.1 405 Method Not Allowed\r\nContent-Length: 0\r\n\r\n')
}
[inline]
pub fn (r mut Response) http_500() {
r.buf += cpy_str(r.buf, 'HTTP/1.1 500 Internal Server Error\r\nContent-Length: 0\r\n\r\n')
}
[inline]
pub fn (r mut Response) raw(response string) {
r.buf += cpy_str(r.buf, response)
}
[inline]
pub fn (r mut Response) end() int {
n := int(r.buf - r.buf_start)
if C.write(r.fd, r.buf_start, n) != n {
return -1
}
return n
}