Commit Graph

62 Commits

Author SHA1 Message Date
Chocobozzz a13df19d57
Increase max image/caption/torrent upload size 2021-06-08 10:01:50 +02:00
Chocobozzz d795b76c44
Fix nginx config 2021-05-11 08:42:13 +02:00
Chocobozzz f2b5aa590e
Update nginx config for resumable upload 2021-05-10 15:27:29 +02:00
kontrollanten f6d6e7f861
Resumable video uploads (#3933)
* WIP: resumable video uploads

relates to #324

* fix review comments

* video upload: error handling

* fix audio upload

* fixes after self review

* Update server/controllers/api/videos/index.ts

Co-authored-by: Rigel Kent <par@rigelk.eu>

* Update server/middlewares/validators/videos/videos.ts

Co-authored-by: Rigel Kent <par@rigelk.eu>

* Update server/controllers/api/videos/index.ts

Co-authored-by: Rigel Kent <par@rigelk.eu>

* update after code review

* refactor upload route

- restore multipart upload route
- move resumable to dedicated upload-resumable route
- move checks to middleware
- do not leak internal fs structure in response

* fix yarn.lock upon rebase

* factorize addVideo for reuse in both endpoints

* add resumable upload API to openapi spec

* add initial test and test helper for resumable upload

* typings for videoAddResumable middleware

* avoid including aws and google packages via node-uploadx, by only including uploadx/core

* rename ex-isAudioBg to more explicit name mentioning it is a preview file for audio

* add video-upload-tmp-folder-cleaner job

* stronger typing of video upload middleware

* reduce dependency to @uploadx/core

* add audio upload test

* refactor resumable uploads cleanup from job to scheduler

* refactor resumable uploads scheduler to compare to last execution time

* make resumable upload validator to always cleanup on failure

* move legacy upload request building outside of uploadVideo test helper

* filter upload-resumable middlewares down to POST, PUT, DELETE

also begin to type metadata

* merge add duration functions

* stronger typings and documentation for uploadx behaviour, move init validator up

* refactor(client/video-edit): options > uploadxOptions

* refactor(client/video-edit): remove obsolete else

* scheduler/remove-dangling-resum: rename tag

* refactor(server/video): add UploadVideoFiles type

* refactor(mw/validators): restructure eslint disable

* refactor(mw/validators/videos): rename import

* refactor(client/vid-upload): rename html elem id

* refactor(sched/remove-dangl): move fn to method

* refactor(mw/async): add method typing

* refactor(mw/vali/video): double quote > single

* refactor(server/upload-resum): express use > all

* proper http methud enum server/middlewares/async.ts

* properly type http methods

* factorize common video upload validation steps

* add check for maximum partially uploaded file size

* fix audioBg use

* fix extname(filename) in addVideo

* document parameters for uploadx's resumable protocol

* clear META files in scheduler

* last audio refactor before cramming preview in the initial POST form data

* refactor as mulitpart/form-data initial post request

this allows preview/thumbnail uploads alongside the initial request,
and cleans up the upload form

* Add more tests for resumable uploads

* Refactor remove dangling resumable uploads

* Prepare changelog

* Add more resumable upload tests

* Remove user quota check for resumable uploads

* Fix upload error handler

* Update nginx template for upload-resumable

* Cleanup comment

* Remove unused express methods

* Prefer to use got instead of raw http

* Don't retry on error 500

Co-authored-by: Rigel Kent <par@rigelk.eu>
Co-authored-by: Rigel Kent <sendmemail@rigelk.eu>
Co-authored-by: Chocobozzz <me@florianbigard.com>
2021-05-10 11:13:41 +02:00
Chocobozzz 9817060fb7
Use new doc website links 2021-02-12 10:12:19 +01:00
Chocobozzz 6c7317a0a3
Fix nginx for small uploads 2021-02-12 10:04:18 +01:00
Rigel Kent e01146559a
fix nginx units 2021-01-31 22:17:37 +01:00
Rigel Kent a59db27090
take into account mime-encoding in nginx client_max_body_size
fixes #3656
2021-01-28 16:15:45 +01:00
Rigel Kent df7b786f09 add `proxy_ignore_client_abort` flag to Nginx conf
The `proxy_ignore_client_abort` flag specifies whether nginx will
monitor possible connection close while waiting for an upstream
server response. If an error occurs while sending a response, the
connection will be closed regardless of the flag, much like if
there were no nginx at all.

fixes #3484
2021-01-25 17:15:04 +01:00
Rigel Kent 111e6c1807
add routes to 2MB cap in nginx config
fixes #3611
2021-01-17 17:00:12 +01:00
Rigel Kent dbe3040e36 merge optional modules within standard ones in support/nginx/peertube 2021-01-13 09:17:13 +01:00
Timo Gurr d168661db3 Mention additional nginx modules required since (#3313)
5f59cf077f introduced requirements on additional nginx modules:

nginx: [emerg] "aio threads" is unsupported on this platform in /etc/nginx/sites-enabled/peertube:247
https://nginx.org/en/docs/http/ngx_http_core_module.html#aio

nginx: [emerg] unknown directive "deny" in /etc/nginx/sites-enabled/peertube:83
https://nginx.org/en/docs/http/ngx_http_access_module.html
2021-01-13 09:17:13 +01:00
Chocobozzz afd4ee86dd
Remove traefik docker support 2020-12-30 17:06:21 +01:00
Rigel Kent d4132d3f56 more explicit error messages for file uploads 2020-12-03 10:15:49 +01:00
Rigel Kent b2aecc1ecb
factorize nginx websocket and per route limits 2020-11-23 10:03:48 +01:00
Rigel Kent 11c449eb1e
remove nginx fd caching, add module requirements 2020-11-18 08:15:55 +01:00
Rigel Kent 901c36d5f4
fix internal redirection cycle whithin nginx client-override optimization 2020-11-17 11:40:14 +01:00
Rigel Kent 5f59cf077f
refresh nginx config and optimize delivery (#3313)
refactors the Nginx configuration for the following points:
- update tls version to include 1.3 by default. so far it was not included by default to make room for previous versions of Nginx, but since 2018 Debian stable has included Nginx in version 1.14.1, and tls 1.3 is available since Nginx 1.13.0.
- clearly indicate that new minimum required version.
- update outdated ssl_ciphers to remove cipher required to support android 4.4, since that version is unsupported since March 2020.
- reordered configuration in sections for easier maintenance: performance optimizations are separated from the vital application/websocket parts.
- move parts that always require manual configuration at the top: peertube host and server name, use server_name 
- move peertube host to a more flexible upstream block: it allows to configure it in one place instead of 3, and is future-proof regarding load-balancing.
- simplified port 80 block: Let’s Encrypt supports 301 redirects.
- group certificate-related config together.
- remove reslover config: it defaults to /etc/resolv.conf which is more than enough.
- align values with their neighbors for easier reading
- always specify units
- always specify default values when they differ from the values set
- use ’m’ for minutes, ’M’ for megabytes
- add consensual optimizations wrt file serving:
  - add timeout optimizations
  - add file descriptor cache optimizations
  - enable sendfile with chunk size > rate limit
  - enable threading
  - tcp optimizations
  - point to further, more system-specific optimizations in the section description
- CDN configuration reduced to one line change
2020-11-16 19:16:49 +01:00
Kimsible 8872828d59
add client overrides to nginx configuration (#3297)
Co-authored-by: Rigel Kent <par@rigelk.eu>
Co-authored-by: kimsible <kimsible@users.noreply.github.com>
2020-11-16 10:34:05 +01:00
Rigel Kent 1a9b141d83
Add nginx behind traefik in docker-compose + image updates
- support/docker/production/docker-compose.yml: addition of a nginx
image reusing support/nginx/peertube nginx conf to improve performance,
and lessen setup differences between the docker-compose install and the
typical production install.
- support/docker/production/docker-compose.yml: postgres 10 -> postgres
12, redis 4 -> redis 5. Postgres major updates implies manual upgrade.
- support/nginx/peertube: HTTP -> HTTPS redirection is now commented
by default, to allow its reuse in support/docker/production/docker-compose.yml.
2020-11-11 16:45:46 +01:00
Rigel Kent 61b20252a0 Add Nginx configuration to redirect videos to an s3 bucket 2020-03-09 09:41:29 +01:00
Rigel Kent 62df8cc1f0
Reword HSTS configuration to warn user of nginx's add_header shortcomings 2020-03-03 11:40:45 +01:00
Chocobozzz 1a5fd848b4
Update nginx cipher to the one we use on framatube 2019-12-17 09:46:28 +01:00
Markus Richter 729c0f4d41 Slightly relax Cipher Suite hardening
This enables legacy software like apps on android 4.4.2 and matches the traefik configuration, where the specific cipher suite is already allowed.
2019-12-12 10:03:56 +01:00
Chocobozzz 63247475a1
Fix nginx config
CORS headers were removed. See https://www.nginx.com/resources/wiki/start/topics/depth/ifisevil/
2019-12-10 16:39:22 +01:00
Chocobozzz f37db8966d
Add streaming playlists endpoint in nginx 2019-12-10 12:11:20 +01:00
Chocobozzz 7b7d4e2a6a
Add client_body_temp_path hint in nginx template 2019-12-05 11:25:00 +01:00
Chocobozzz fd2ddcae8f
Fix nginx template on dual stack server
See https://framacolibri.org/t/listen-to-unix-socket-instead-of-localhost-9000/5348
2019-08-07 15:13:01 +02:00
Chocobozzz c928e1364f
Improve nginx client images cache 2019-07-29 14:58:41 +02:00
Benjamin Bouvier 7eeb6a0ba4 Nginx config file: remove text/html from gzip_types
As stated by https://nginx.org/en/docs/http/ngx_http_gzip_module.html, text/html is always part of the gzip_types. This removes a warning when checking the Nginx configuration files.
2019-02-11 04:29:51 +01:00
Chocobozzz 4a57b65cc5
Support socket.io in nginx template 2019-01-29 09:10:24 +01:00
Micah Elizabeth Scott 4b49385892 Remove hard-coded 8GB upload limit in client (#1293)
* Remove hard-coded 8GB upload limit in client

Ideally we'd know what the specific server's configured upload limit
is before starting, but this 8GB limit is not useful if an administrator
has changed the nginx post limit on the server.

* Better docs for admins about client_max_body_size

Seems like some admins already tweak this value up or down to allow
for different maximum video upload sizes. The current codebase has no
other server-side limits that I'm aware of, and I've been routinely
uploading quite large videos to my instance.

This patch replaces the somewhat incorrect (or outdated?) 'hard limit'
comment with some advice about allocating enough space for nginx and
communicating the limit with your users.

Of course it would be better if this configuration could be unified with
PeerTube's config somehow. I'm not sure whether the best option there is
to turn off nginx's buffering here and let PeerTube handle the entire upload
(can we do this only for the video upload API endpoint?) or whether we want
PeerTube to generate nginx configs in a more automated way layer. In any case,
this patch is intended as an incremental improvement.
2018-12-07 14:58:17 +01:00
Chocobozzz b9fffa297f
Create redundancy endpoint 2018-12-04 17:08:55 +01:00
Chocobozzz 415acc63cf
Add comments in nginx regarding blocks that can be safely removed 2018-09-17 17:45:54 +02:00
Felix Ableitner 5284d4028c Don't include `preload` flag in sample HSTS header
This goes against the recommendations (preloading should be opt-in). Putting it in the example makes it likely that people enable it without knowing what it means.

https://hstspreload.org/?domain=peertube.social#opt-in
2018-09-11 20:10:57 +02:00
Rigel Kent 6328da8c01
make HSTS opt-in and leave it to the reverse-proxy 2018-09-09 22:10:38 +02:00
Micah Elizabeth Scott a18e02f358 Only enable gzip for HTML/CSS/JS
No compression on JSON endpoints, in order to protect
from potential compression+encryption data leak attacks (like BREACH)
2018-08-24 09:08:33 +02:00
Micah Elizabeth Scott b9ad995605 Add gzip support to the sample nginx configuration
Without gzip explicitly enabled, load times suffer from transferring
over a megabyte of plaintext javascript. With gzip enabled, the bundle
is down to about 300K, and loads much faster.

This change does not enable gzip on files that are already compressed,
so images, fonts, and videos will be sent without the CPU overhead.
2018-08-24 09:08:33 +02:00
Chocobozzz a8bf1d826e
404 on unknown thumbnail 2018-07-24 18:03:40 +02:00
Chocobozzz 7f8db30ccd
Add cors to static route in nginx template 2018-07-24 18:03:39 +02:00
Rigel Kent 828fdd08b7
(nginx) remove headers now dealt with helmet 2018-07-18 10:21:59 +02:00
Chocobozzz 57a81ff649
Fix static avatars/thumbnails cache 2018-07-17 19:04:41 +02:00
Chocobozzz 34b1919290
Increase upload limit to 8GB (test) 2018-06-29 17:10:53 +02:00
Chocobozzz 051bf3f773
Revert "Selective route permission to use embeds, fixes #322 in a better way (#364)" (#365)
This reverts commit d40cd86bf5.
2018-03-20 17:39:36 +01:00
Rigel Kent d40cd86bf5 Selective route permission to use embeds, fixes #322 in a better way (#364) 2018-03-20 17:28:41 +01:00
Valvin 446f78d7b4 Remove X-Frame options in nginx config (#322)
`X-Frame-Options DENY;` doesn't permit sharing using iframe
2018-03-05 09:40:36 +01:00
Chocobozzz 4919b6304f
Fix nginx configuration that do not work with import-videos script 2018-03-01 17:14:57 +01:00
Chocobozzz 0b49571268
Try to improve production guide 2018-02-16 11:04:12 +01:00
Rigel Kent e883399fa6 Precisions and security enhancements to the production guide (#287)
- added precisions and suggestions about how to generate Let's Encrypt certificates. Users have reported their installations didn't work when the problem came from missing certificates (false positives).
- security defaults of Nginx follow the basic robustness principle "be conservative in what you send, be liberal in what you accept", which isn't enough with modern security standards, so we should be picky with the cipher suites we use, among other things. Extra comments (especially for the TLS1.3 protocol support parameter) make the requirement of a recent Nginx installation obvious, and the downgrade alternative remains clear to the system administrator.

All in all, we should aknowledge users will most often copy and paste the configuration files. Making them secure by default may force a few users to read their configuration, but on the long run we are making the fediverse more secure.

Since I've come to modify a bit the Nginx config in `support/doc/production.md`, I've merged it with the template so that they stay consistent.
2018-02-14 11:11:49 +01:00
Chocobozzz 59c48d49c5
Peertube home in /var/www instead of /home 2018-01-23 09:00:23 +01:00