2019-06-25 12:06:55 +02:00
|
|
|
## Code Structure
|
|
|
|
|
2020-03-15 09:18:05 +01:00
|
|
|
I tried to make the code of the compiler and vlib as simple and readable as
|
|
|
|
possible. One of V's goals is to be open to developers with different levels
|
|
|
|
of experience in compiler development. Compilers don't need to be black boxes
|
|
|
|
full of magic that only few people understand.
|
2019-06-25 12:06:55 +02:00
|
|
|
|
2020-04-14 15:13:36 +02:00
|
|
|
The V compiler is modular, and can be used by other applications. It is located
|
|
|
|
in `cmd/v/` and `vlib/v/`.
|
2019-06-25 12:06:55 +02:00
|
|
|
|
2020-05-29 03:05:04 +02:00
|
|
|
The most important and useful command to remember when working on the V compiler
|
|
|
|
is `v self`.
|
2020-04-17 18:20:05 +02:00
|
|
|
It rebuilds the V compiler.
|
|
|
|
|
2020-05-29 03:05:04 +02:00
|
|
|
Be careful, if you introduce a breaking change and rebuild V, you will no longer
|
|
|
|
be able to use V to build itself. So it's a good idea to make a backup copy of a
|
|
|
|
working compiler executable.
|
2020-04-17 18:20:05 +02:00
|
|
|
|
2020-09-07 13:48:15 +02:00
|
|
|
But don't worry, you can always simply run `make` (or `make.bat`), it will
|
|
|
|
download the C version of the compiler and rebuild it from scratch.
|
2020-04-17 18:20:05 +02:00
|
|
|
|
2020-11-12 09:32:46 +01:00
|
|
|
The architecture of the compiler is very simple and has three distinct steps:
|
|
|
|
|
2021-02-11 10:04:44 +01:00
|
|
|
Parse/generate AST (`v.parser`) => Check types (`v.checker`)
|
2020-11-16 16:49:40 +01:00
|
|
|
=> Generate C/JavaScript/machine code (`v.gen`)
|
2020-11-12 09:32:46 +01:00
|
|
|
|
2020-04-17 18:20:05 +02:00
|
|
|
|
2019-09-15 14:57:17 +02:00
|
|
|
The main files are:
|
2019-06-25 12:06:55 +02:00
|
|
|
|
2020-11-12 09:32:46 +01:00
|
|
|
1. `cmd/v/v.v` The entry point.
|
2019-09-29 12:58:37 +02:00
|
|
|
|
2019-06-25 12:06:55 +02:00
|
|
|
- V figures out the build mode.
|
|
|
|
- Constructs the compiler object (`struct V`).
|
|
|
|
- Creates a list of .v files that need to be parsed.
|
2020-03-15 09:18:05 +01:00
|
|
|
- Creates a parser object for each file and runs `parse()` on them.
|
2021-04-26 15:39:38 +02:00
|
|
|
- The correct backend is called (C, JS, native), and a binary is compiled.
|
2019-06-25 12:06:55 +02:00
|
|
|
|
2020-03-15 09:21:31 +01:00
|
|
|
2. `v/scanner` The scanner's job is to parse a list of characters and convert
|
2020-03-15 09:18:05 +01:00
|
|
|
them to tokens.
|
2019-09-15 14:57:17 +02:00
|
|
|
|
2020-03-15 09:21:31 +01:00
|
|
|
3. `v/token` This is simply a list of all tokens, their string values, and a
|
2020-03-15 09:18:05 +01:00
|
|
|
couple of helper functions.
|
2019-06-25 12:06:55 +02:00
|
|
|
|
2020-03-15 09:21:31 +01:00
|
|
|
4. `v/parser` The parser. It converts a list of tokens into an AST.
|
2020-05-29 03:05:04 +02:00
|
|
|
In V, objects can be used before declaration, so unknown types are marked as
|
|
|
|
unresolved. They are resolved later in the type checker.
|
2020-03-15 09:21:31 +01:00
|
|
|
|
2020-03-15 09:18:05 +01:00
|
|
|
5. `v/table` V creates one table object that is shared by all parsers. It
|
|
|
|
contains all types, consts, and functions, as well as several helpers to search
|
|
|
|
for objects by name, register new objects, modify types' fields, etc.
|
2019-06-25 12:06:55 +02:00
|
|
|
|
2020-11-12 09:32:46 +01:00
|
|
|
6. `v/checker` Type checker and resolver. It processes the AST and makes sure
|
2020-03-15 09:18:05 +01:00
|
|
|
the types are correct. Unresolved types are resolved, type information is added
|
|
|
|
to the AST.
|
2019-06-25 12:06:55 +02:00
|
|
|
|
2021-02-11 10:04:44 +01:00
|
|
|
7. `v/gen/c` C backend. It simply walks the AST and generates C code that can be
|
2020-03-15 09:18:05 +01:00
|
|
|
compiled with Clang, GCC, Visual Studio, and TCC.
|
2019-06-25 12:06:55 +02:00
|
|
|
|
2020-03-15 09:18:05 +01:00
|
|
|
8. `json.v` defines the json code generation. This file will be removed once V
|
|
|
|
supports comptime code generation, and it will be possible to do this using the
|
|
|
|
language's tools.
|
2019-06-25 12:06:55 +02:00
|
|
|
|
2021-04-26 15:39:38 +02:00
|
|
|
9. `v/gen/native` is the directory with all the machine code generation logic. It
|
2020-03-15 09:18:05 +01:00
|
|
|
defines a set of functions that translate assembly instructions to machine code
|
|
|
|
and build the binary from scratch byte by byte. It manually builds all headers,
|
|
|
|
segments, sections, symtable, relocations, etc. Right now it only has basic
|
2021-04-26 15:39:38 +02:00
|
|
|
support of the native platform (ELF, MACHO format).
|
2019-06-25 12:06:55 +02:00
|
|
|
|
2020-03-15 09:18:05 +01:00
|
|
|
The rest of the directories are vlib modules: `builtin/` (strings, arrays,
|
|
|
|
maps), `time/`, `os/`, etc. Their documentation is pretty clear.
|
2019-06-25 12:06:55 +02:00
|
|
|
|
2020-03-15 09:21:31 +01:00
|
|
|
## Example Workflow for Contributing
|
|
|
|
(provided by [@spytheman](https://github.com/spytheman))
|
2019-10-04 22:06:10 +02:00
|
|
|
|
2020-03-15 09:18:05 +01:00
|
|
|
(If you don't already have a GitHub account, please create one. Your GitHub
|
|
|
|
username will be referred to later as 'YOUR_GITHUB_USERNAME'. Change it
|
|
|
|
accordingly in the steps below.)
|
2019-10-04 22:06:10 +02:00
|
|
|
|
2020-03-27 16:03:06 +01:00
|
|
|
1. Fork https://github.com/vlang/v using GitHub's interface to your own account.
|
|
|
|
Let's say that the forked repository is at
|
|
|
|
`https://github.com/YOUR_GITHUB_USERNAME/v` .
|
2020-05-29 03:05:04 +02:00
|
|
|
2. Clone the main v repository https://github.com/vlang/v to a local folder on
|
|
|
|
your computer, say named nv/ (`git clone https://github.com/vlang/v nv`)
|
2020-03-27 16:03:06 +01:00
|
|
|
3. `cd nv`
|
2020-04-14 15:13:36 +02:00
|
|
|
4. `git remote add pullrequest https://github.com/YOUR_GITHUB_USERNAME/v`
|
2020-05-29 03:05:04 +02:00
|
|
|
NB: the remote named `pullrequest` should point to YOUR own forked repo, not the
|
|
|
|
main v repository! After this, your local cloned repository is prepared for
|
|
|
|
making pullrequests, and you can just do normal git operations such as:
|
|
|
|
`git pull` `git status` and so on.
|
2020-03-27 16:03:06 +01:00
|
|
|
|
2020-04-14 15:13:36 +02:00
|
|
|
5. When finished with a feature/bugfix/change, you can:
|
2020-03-27 16:03:06 +01:00
|
|
|
`git checkout -b fix_alabala`
|
2020-04-14 15:13:36 +02:00
|
|
|
6. `git push pullrequest` # (NOTE: the `pullrequest` remote was setup on step 4)
|
|
|
|
7. On GitHub's web interface, go to: https://github.com/vlang/v/pulls
|
2020-03-27 16:03:06 +01:00
|
|
|
|
2020-07-01 13:36:08 +02:00
|
|
|
Here the UI shows a dialog with a button to make a new pull request based on
|
|
|
|
the new pushed branch.
|
|
|
|
(Example dialog: https://url4e.com/gyazo/images/364edc04.png)
|
|
|
|
|
2020-04-14 15:13:36 +02:00
|
|
|
8. After making your pullrequest (aka, PR), you can continue to work on the
|
2020-05-29 03:05:04 +02:00
|
|
|
branch `fix_alabala` ... just do again `git push pullrequest` when you have more
|
|
|
|
commits.
|
2020-07-01 13:36:08 +02:00
|
|
|
|
2020-04-14 15:13:36 +02:00
|
|
|
9. If there are merge conflicts, or a branch lags too much behind V's master,
|
2020-03-15 09:18:05 +01:00
|
|
|
you can do the following:
|
2020-07-01 13:36:08 +02:00
|
|
|
|
2020-05-29 03:05:04 +02:00
|
|
|
1. `git pull --rebase origin master` # solve conflicts and do
|
|
|
|
`git rebase --continue`
|
|
|
|
2. `git push pullrequest -f` # this will overwrite your current remote branch
|
|
|
|
with the updated version of your changes.
|
2020-03-27 16:03:06 +01:00
|
|
|
|
|
|
|
The point of doing the above steps, is to never directly push to the main V
|
|
|
|
repository, *only to your own fork*. Since your local `master` branch tracks the
|
|
|
|
main V repository's master, then `git checkout master`, as well as
|
|
|
|
`git pull --rebase origin master` will continue to work as expected
|
|
|
|
(these are actually used by `v up`) and git can always do it cleanly.
|
|
|
|
|
|
|
|
Git is very flexible, so there are other ways to accomplish the same thing.
|
2020-07-05 07:46:42 +02:00
|
|
|
See the [GitHub flow](https://guides.github.com/introduction/git-handbook/#github), for more information.
|
2020-03-27 16:03:06 +01:00
|
|
|
|
|
|
|
## Using Github's hub CLI tool
|
|
|
|
|
|
|
|
You can download the `hub` tool from https://hub.github.com/ . Using
|
2020-04-14 15:13:36 +02:00
|
|
|
`hub`, you will not need to go through the (sometimes) slow website
|
2020-03-27 16:03:06 +01:00
|
|
|
to make PRs. Most remote operations can be done through the `hub` CLI
|
|
|
|
command.
|
|
|
|
|
|
|
|
NB: You still need to have a GitHub account.
|
|
|
|
|
|
|
|
### Preparation:
|
|
|
|
(steps 1..3 need to be done just *once*):
|
|
|
|
|
|
|
|
1. `hub clone vlang/v my_v`
|
|
|
|
2. `cd my_v`
|
|
|
|
3. `hub fork --remote-name pullrequest`
|
|
|
|
|
2020-05-29 03:05:04 +02:00
|
|
|
4. `git checkout -b my_cool_feature` # Step 4 is better done *once per each new
|
|
|
|
feature/bugfix* that you make.
|
2020-03-27 16:03:06 +01:00
|
|
|
|
|
|
|
### Improve V by making commits:
|
|
|
|
|
|
|
|
5. `git commit -am "math: add a new function copysign"`
|
|
|
|
|
|
|
|
### Testing your commits locally:
|
2020-04-14 15:13:36 +02:00
|
|
|
You can test locally whether your changes have not broken something by
|
2021-01-17 15:04:08 +01:00
|
|
|
running: `v test-all`. See `TESTS.md` for more details.
|
2020-03-27 16:03:06 +01:00
|
|
|
|
|
|
|
### Publishing your commits to GitHub:
|
|
|
|
|
|
|
|
6. `git push pullrequest`
|
|
|
|
|
|
|
|
### Making a PR with `hub`:
|
|
|
|
(so that your changes can be merged to the main V repository)
|
|
|
|
|
|
|
|
7. `hub pull-request`
|
|
|
|
|
|
|
|
Optionally, you can track the status of your PR CI tests with:
|
|
|
|
|
|
|
|
8. `hub ci-status --verbose`
|
|
|
|
|
|
|
|
### Fixing failing tests:
|
|
|
|
If everything is OK, after 5-10 minutes, the CI tests should pass for
|
|
|
|
all platforms. If not, visit the URLs for the failing CI jobs, see
|
2020-04-14 15:13:36 +02:00
|
|
|
which tests have failed and then fix them by making more changes. Just use
|
2020-03-27 16:03:06 +01:00
|
|
|
`git push pullrequest` to publish your changes. The CI tests will
|
|
|
|
run with your updated code. Use `hub ci-status --verbose` to monitor
|
|
|
|
their status.
|
2021-01-10 15:27:17 +01:00
|
|
|
|
|
|
|
## Flags
|
|
|
|
|
|
|
|
V allows you to pass custom flags using `-d my_flag` that can then be checked
|
|
|
|
at compile time (see the documentation about
|
|
|
|
[compile-time if](https://github.com/vlang/v/blob/master/doc/docs.md#compile-time-if)).
|
2021-02-11 10:04:44 +01:00
|
|
|
There are numerous flags that can be passed when building the compiler
|
|
|
|
with `v self` or when creating a copy of the compiler, that will help
|
2021-01-10 15:27:17 +01:00
|
|
|
you when debugging.
|
|
|
|
|
2021-02-11 10:04:44 +01:00
|
|
|
Beware that the flags must be passed when building the compiler,
|
2021-01-10 15:27:17 +01:00
|
|
|
not the program, so do for example: `v -d time_parsing cmd/v` or
|
|
|
|
`v -d trace_checker self`.
|
|
|
|
Some flags can make the compiler very verbose, so it is recommended
|
|
|
|
to create a copy of the compiler rather than replacing it with `v self`.
|
|
|
|
|
|
|
|
| Flag | Usage |
|
|
|
|
|------|-------|
|
|
|
|
| `debugautostr` | Prints informations about `.str()` method auto-generated by the compiler during C generation |
|
|
|
|
| `debugscanner` | Prints debug information during the scanning phase |
|
|
|
|
| `debug_codegen` | Prints automatically generated V code during the scanning phase |
|
|
|
|
| `debug_interface_table` | Prints generated interfaces during C generation |
|
|
|
|
| `debug_interface_type_implements` | Prints debug information when checking that a type implements in interface |
|
|
|
|
| `print_vweb_template_expansions` | Prints vweb compiled HTML files |
|
|
|
|
| `time_checking` | Prints the time spent checking files and other related informations |
|
|
|
|
| `time_parsing` | Prints the time spent parsing files and other related informations |
|
|
|
|
| `trace_ccoptions` | Prints options passed down to the C compiler |
|
|
|
|
| `trace_checker` | Prints informations about the statements being checked |
|
|
|
|
| `trace_gen` | Prints strings written to the generated C file. Beware, this flag is very verbose |
|
|
|
|
| `trace_parser` | Prints informations about parsed statements and expressions |
|
|
|
|
| `trace_thirdparty_obj_files` | Prints informations about built thirdparty obj files |
|
|
|
|
| `trace_use_cache` | Prints informations about cache use |
|