2019-06-25 12:06:55 +02:00
|
|
|
## Code Structure
|
|
|
|
|
2020-03-15 09:18:05 +01:00
|
|
|
I tried to make the code of the compiler and vlib as simple and readable as
|
|
|
|
possible. One of V's goals is to be open to developers with different levels
|
|
|
|
of experience in compiler development. Compilers don't need to be black boxes
|
|
|
|
full of magic that only few people understand.
|
2019-06-25 12:06:55 +02:00
|
|
|
|
2020-04-14 15:13:36 +02:00
|
|
|
The V compiler is modular, and can be used by other applications. It is located
|
|
|
|
in `cmd/v/` and `vlib/v/`.
|
2019-06-25 12:06:55 +02:00
|
|
|
|
2020-05-29 03:05:04 +02:00
|
|
|
The most important and useful command to remember when working on the V compiler
|
|
|
|
is `v self`.
|
2020-04-17 18:20:05 +02:00
|
|
|
It rebuilds the V compiler.
|
|
|
|
|
2020-05-29 03:05:04 +02:00
|
|
|
Be careful, if you introduce a breaking change and rebuild V, you will no longer
|
|
|
|
be able to use V to build itself. So it's a good idea to make a backup copy of a
|
|
|
|
working compiler executable.
|
2020-04-17 18:20:05 +02:00
|
|
|
|
2020-09-07 13:48:15 +02:00
|
|
|
But don't worry, you can always simply run `make` (or `make.bat`), it will
|
|
|
|
download the C version of the compiler and rebuild it from scratch.
|
2020-04-17 18:20:05 +02:00
|
|
|
|
2020-11-12 09:32:46 +01:00
|
|
|
The architecture of the compiler is very simple and has three distinct steps:
|
|
|
|
|
2020-11-16 16:49:40 +01:00
|
|
|
Parse/generate AST (`v.parser`) => Check types (`v.checker`)
|
|
|
|
=> Generate C/JavaScript/machine code (`v.gen`)
|
2020-11-12 09:32:46 +01:00
|
|
|
|
2020-04-17 18:20:05 +02:00
|
|
|
|
2019-09-15 14:57:17 +02:00
|
|
|
The main files are:
|
2019-06-25 12:06:55 +02:00
|
|
|
|
2020-11-12 09:32:46 +01:00
|
|
|
1. `cmd/v/v.v` The entry point.
|
2019-09-29 12:58:37 +02:00
|
|
|
|
2019-06-25 12:06:55 +02:00
|
|
|
- V figures out the build mode.
|
|
|
|
- Constructs the compiler object (`struct V`).
|
|
|
|
- Creates a list of .v files that need to be parsed.
|
2020-03-15 09:18:05 +01:00
|
|
|
- Creates a parser object for each file and runs `parse()` on them.
|
|
|
|
- The correct backend is called (C, JS, x64), and a binary is compiled.
|
2019-06-25 12:06:55 +02:00
|
|
|
|
2020-03-15 09:21:31 +01:00
|
|
|
2. `v/scanner` The scanner's job is to parse a list of characters and convert
|
2020-03-15 09:18:05 +01:00
|
|
|
them to tokens.
|
2019-09-15 14:57:17 +02:00
|
|
|
|
2020-03-15 09:21:31 +01:00
|
|
|
3. `v/token` This is simply a list of all tokens, their string values, and a
|
2020-03-15 09:18:05 +01:00
|
|
|
couple of helper functions.
|
2019-06-25 12:06:55 +02:00
|
|
|
|
2020-03-15 09:21:31 +01:00
|
|
|
4. `v/parser` The parser. It converts a list of tokens into an AST.
|
2020-05-29 03:05:04 +02:00
|
|
|
In V, objects can be used before declaration, so unknown types are marked as
|
|
|
|
unresolved. They are resolved later in the type checker.
|
2020-03-15 09:21:31 +01:00
|
|
|
|
2020-03-15 09:18:05 +01:00
|
|
|
5. `v/table` V creates one table object that is shared by all parsers. It
|
|
|
|
contains all types, consts, and functions, as well as several helpers to search
|
|
|
|
for objects by name, register new objects, modify types' fields, etc.
|
2019-06-25 12:06:55 +02:00
|
|
|
|
2020-11-12 09:32:46 +01:00
|
|
|
6. `v/checker` Type checker and resolver. It processes the AST and makes sure
|
2020-03-15 09:18:05 +01:00
|
|
|
the types are correct. Unresolved types are resolved, type information is added
|
|
|
|
to the AST.
|
2019-06-25 12:06:55 +02:00
|
|
|
|
2020-03-15 09:18:05 +01:00
|
|
|
7. `v/gen` C backend. It simply walks the AST and generates C code that can be
|
|
|
|
compiled with Clang, GCC, Visual Studio, and TCC.
|
2019-06-25 12:06:55 +02:00
|
|
|
|
2020-03-15 09:18:05 +01:00
|
|
|
8. `json.v` defines the json code generation. This file will be removed once V
|
|
|
|
supports comptime code generation, and it will be possible to do this using the
|
|
|
|
language's tools.
|
2019-06-25 12:06:55 +02:00
|
|
|
|
2020-03-15 09:18:05 +01:00
|
|
|
9. `v/gen/x64` is the directory with all the machine code generation logic. It
|
|
|
|
defines a set of functions that translate assembly instructions to machine code
|
|
|
|
and build the binary from scratch byte by byte. It manually builds all headers,
|
|
|
|
segments, sections, symtable, relocations, etc. Right now it only has basic
|
|
|
|
support of the x64 platform/ELF format.
|
2019-06-25 12:06:55 +02:00
|
|
|
|
2020-03-15 09:18:05 +01:00
|
|
|
The rest of the directories are vlib modules: `builtin/` (strings, arrays,
|
|
|
|
maps), `time/`, `os/`, etc. Their documentation is pretty clear.
|
2019-06-25 12:06:55 +02:00
|
|
|
|
2020-03-15 09:21:31 +01:00
|
|
|
## Example Workflow for Contributing
|
|
|
|
(provided by [@spytheman](https://github.com/spytheman))
|
2019-10-04 22:06:10 +02:00
|
|
|
|
2020-03-15 09:18:05 +01:00
|
|
|
(If you don't already have a GitHub account, please create one. Your GitHub
|
|
|
|
username will be referred to later as 'YOUR_GITHUB_USERNAME'. Change it
|
|
|
|
accordingly in the steps below.)
|
2019-10-04 22:06:10 +02:00
|
|
|
|
2020-03-27 16:03:06 +01:00
|
|
|
1. Fork https://github.com/vlang/v using GitHub's interface to your own account.
|
|
|
|
Let's say that the forked repository is at
|
|
|
|
`https://github.com/YOUR_GITHUB_USERNAME/v` .
|
2020-05-29 03:05:04 +02:00
|
|
|
2. Clone the main v repository https://github.com/vlang/v to a local folder on
|
|
|
|
your computer, say named nv/ (`git clone https://github.com/vlang/v nv`)
|
2020-03-27 16:03:06 +01:00
|
|
|
3. `cd nv`
|
2020-04-14 15:13:36 +02:00
|
|
|
4. `git remote add pullrequest https://github.com/YOUR_GITHUB_USERNAME/v`
|
2020-05-29 03:05:04 +02:00
|
|
|
NB: the remote named `pullrequest` should point to YOUR own forked repo, not the
|
|
|
|
main v repository! After this, your local cloned repository is prepared for
|
|
|
|
making pullrequests, and you can just do normal git operations such as:
|
|
|
|
`git pull` `git status` and so on.
|
2020-03-27 16:03:06 +01:00
|
|
|
|
2020-04-14 15:13:36 +02:00
|
|
|
5. When finished with a feature/bugfix/change, you can:
|
2020-03-27 16:03:06 +01:00
|
|
|
`git checkout -b fix_alabala`
|
2020-04-14 15:13:36 +02:00
|
|
|
6. `git push pullrequest` # (NOTE: the `pullrequest` remote was setup on step 4)
|
|
|
|
7. On GitHub's web interface, go to: https://github.com/vlang/v/pulls
|
2020-03-27 16:03:06 +01:00
|
|
|
|
2020-07-01 13:36:08 +02:00
|
|
|
Here the UI shows a dialog with a button to make a new pull request based on
|
|
|
|
the new pushed branch.
|
|
|
|
(Example dialog: https://url4e.com/gyazo/images/364edc04.png)
|
|
|
|
|
2020-04-14 15:13:36 +02:00
|
|
|
8. After making your pullrequest (aka, PR), you can continue to work on the
|
2020-05-29 03:05:04 +02:00
|
|
|
branch `fix_alabala` ... just do again `git push pullrequest` when you have more
|
|
|
|
commits.
|
2020-07-01 13:36:08 +02:00
|
|
|
|
2020-04-14 15:13:36 +02:00
|
|
|
9. If there are merge conflicts, or a branch lags too much behind V's master,
|
2020-03-15 09:18:05 +01:00
|
|
|
you can do the following:
|
2020-07-01 13:36:08 +02:00
|
|
|
|
2020-05-29 03:05:04 +02:00
|
|
|
1. `git pull --rebase origin master` # solve conflicts and do
|
|
|
|
`git rebase --continue`
|
|
|
|
2. `git push pullrequest -f` # this will overwrite your current remote branch
|
|
|
|
with the updated version of your changes.
|
2020-03-27 16:03:06 +01:00
|
|
|
|
|
|
|
The point of doing the above steps, is to never directly push to the main V
|
|
|
|
repository, *only to your own fork*. Since your local `master` branch tracks the
|
|
|
|
main V repository's master, then `git checkout master`, as well as
|
|
|
|
`git pull --rebase origin master` will continue to work as expected
|
|
|
|
(these are actually used by `v up`) and git can always do it cleanly.
|
|
|
|
|
|
|
|
Git is very flexible, so there are other ways to accomplish the same thing.
|
2020-07-05 07:46:42 +02:00
|
|
|
See the [GitHub flow](https://guides.github.com/introduction/git-handbook/#github), for more information.
|
2020-03-27 16:03:06 +01:00
|
|
|
|
|
|
|
## Using Github's hub CLI tool
|
|
|
|
|
|
|
|
You can download the `hub` tool from https://hub.github.com/ . Using
|
2020-04-14 15:13:36 +02:00
|
|
|
`hub`, you will not need to go through the (sometimes) slow website
|
2020-03-27 16:03:06 +01:00
|
|
|
to make PRs. Most remote operations can be done through the `hub` CLI
|
|
|
|
command.
|
|
|
|
|
|
|
|
NB: You still need to have a GitHub account.
|
|
|
|
|
|
|
|
### Preparation:
|
|
|
|
(steps 1..3 need to be done just *once*):
|
|
|
|
|
|
|
|
1. `hub clone vlang/v my_v`
|
|
|
|
2. `cd my_v`
|
|
|
|
3. `hub fork --remote-name pullrequest`
|
|
|
|
|
2020-05-29 03:05:04 +02:00
|
|
|
4. `git checkout -b my_cool_feature` # Step 4 is better done *once per each new
|
|
|
|
feature/bugfix* that you make.
|
2020-03-27 16:03:06 +01:00
|
|
|
|
|
|
|
### Improve V by making commits:
|
|
|
|
|
|
|
|
5. `git commit -am "math: add a new function copysign"`
|
|
|
|
|
|
|
|
### Testing your commits locally:
|
2020-04-14 15:13:36 +02:00
|
|
|
You can test locally whether your changes have not broken something by
|
2020-09-07 13:48:15 +02:00
|
|
|
running: `v test-compiler`. See `TESTS.md` for more details.
|
2020-03-27 16:03:06 +01:00
|
|
|
|
|
|
|
### Publishing your commits to GitHub:
|
|
|
|
|
|
|
|
6. `git push pullrequest`
|
|
|
|
|
|
|
|
### Making a PR with `hub`:
|
|
|
|
(so that your changes can be merged to the main V repository)
|
|
|
|
|
|
|
|
7. `hub pull-request`
|
|
|
|
|
|
|
|
Optionally, you can track the status of your PR CI tests with:
|
|
|
|
|
|
|
|
8. `hub ci-status --verbose`
|
|
|
|
|
|
|
|
### Fixing failing tests:
|
|
|
|
If everything is OK, after 5-10 minutes, the CI tests should pass for
|
|
|
|
all platforms. If not, visit the URLs for the failing CI jobs, see
|
2020-04-14 15:13:36 +02:00
|
|
|
which tests have failed and then fix them by making more changes. Just use
|
2020-03-27 16:03:06 +01:00
|
|
|
`git push pullrequest` to publish your changes. The CI tests will
|
|
|
|
run with your updated code. Use `hub ci-status --verbose` to monitor
|
|
|
|
their status.
|