Commit Graph

59 Commits

Author SHA1 Message Date
kontrollanten f6d6e7f861
Resumable video uploads (#3933)
* WIP: resumable video uploads

relates to #324

* fix review comments

* video upload: error handling

* fix audio upload

* fixes after self review

* Update server/controllers/api/videos/index.ts

Co-authored-by: Rigel Kent <par@rigelk.eu>

* Update server/middlewares/validators/videos/videos.ts

Co-authored-by: Rigel Kent <par@rigelk.eu>

* Update server/controllers/api/videos/index.ts

Co-authored-by: Rigel Kent <par@rigelk.eu>

* update after code review

* refactor upload route

- restore multipart upload route
- move resumable to dedicated upload-resumable route
- move checks to middleware
- do not leak internal fs structure in response

* fix yarn.lock upon rebase

* factorize addVideo for reuse in both endpoints

* add resumable upload API to openapi spec

* add initial test and test helper for resumable upload

* typings for videoAddResumable middleware

* avoid including aws and google packages via node-uploadx, by only including uploadx/core

* rename ex-isAudioBg to more explicit name mentioning it is a preview file for audio

* add video-upload-tmp-folder-cleaner job

* stronger typing of video upload middleware

* reduce dependency to @uploadx/core

* add audio upload test

* refactor resumable uploads cleanup from job to scheduler

* refactor resumable uploads scheduler to compare to last execution time

* make resumable upload validator to always cleanup on failure

* move legacy upload request building outside of uploadVideo test helper

* filter upload-resumable middlewares down to POST, PUT, DELETE

also begin to type metadata

* merge add duration functions

* stronger typings and documentation for uploadx behaviour, move init validator up

* refactor(client/video-edit): options > uploadxOptions

* refactor(client/video-edit): remove obsolete else

* scheduler/remove-dangling-resum: rename tag

* refactor(server/video): add UploadVideoFiles type

* refactor(mw/validators): restructure eslint disable

* refactor(mw/validators/videos): rename import

* refactor(client/vid-upload): rename html elem id

* refactor(sched/remove-dangl): move fn to method

* refactor(mw/async): add method typing

* refactor(mw/vali/video): double quote > single

* refactor(server/upload-resum): express use > all

* proper http methud enum server/middlewares/async.ts

* properly type http methods

* factorize common video upload validation steps

* add check for maximum partially uploaded file size

* fix audioBg use

* fix extname(filename) in addVideo

* document parameters for uploadx's resumable protocol

* clear META files in scheduler

* last audio refactor before cramming preview in the initial POST form data

* refactor as mulitpart/form-data initial post request

this allows preview/thumbnail uploads alongside the initial request,
and cleans up the upload form

* Add more tests for resumable uploads

* Refactor remove dangling resumable uploads

* Prepare changelog

* Add more resumable upload tests

* Remove user quota check for resumable uploads

* Fix upload error handler

* Update nginx template for upload-resumable

* Cleanup comment

* Remove unused express methods

* Prefer to use got instead of raw http

* Don't retry on error 500

Co-authored-by: Rigel Kent <par@rigelk.eu>
Co-authored-by: Rigel Kent <sendmemail@rigelk.eu>
Co-authored-by: Chocobozzz <me@florianbigard.com>
2021-05-10 11:13:41 +02:00
Chocobozzz 805b8619c1
Fix (again) youtube import 2021-01-19 16:36:21 +01:00
Chocobozzz 8dc8a34ee8 Avoir some circular dependencies 2020-05-04 16:21:39 +02:00
Chocobozzz d57d1d83c6
Support audio files import 2020-04-03 15:41:39 +02:00
Chocobozzz a15871560f
Move to eslint 2020-02-03 08:31:02 +01:00
Chocobozzz 35f28e94c7
Add infohash cache 2020-01-03 13:47:45 +01:00
Chocobozzz b5fecbf441
Type toActivityPubObject functions 2019-08-21 15:23:04 +02:00
Chocobozzz a8b666e9f1 Add plugin static files cache 2019-07-24 10:58:16 +02:00
Chocobozzz 6dd9de95df
Move config in its own file 2019-04-11 13:45:39 +02:00
Chocobozzz 4c280004ce Use a single file instead of segments for HLS 2019-02-11 09:13:02 +01:00
Chocobozzz a4101923e6
Implement contact form on server side 2019-01-10 11:32:37 +01:00
Chocobozzz 6040f87d14
Add tmp and redundancy directories 2018-12-04 16:04:15 +01:00
Chocobozzz 1b5e2d7290
Optimize config endpoint 2018-11-23 11:07:00 +01:00
Chocobozzz 7ad9b9846c Add ability for users to block an account/instance on server side 2018-10-16 16:41:36 +02:00
Felix Ableitner edb4ffc7e0 Set bitrate limits for transcoding (fixes #638) (#1135)
* Set bitrate limits for transcoding (fixes #638)

* added optimization script and test, changed stuff

* fix test, improve docs

* re-add optimize-old-videos script

* added documentation

* Don't optimize videos without valid UUID, or redundancy videos

* move getUUIDFromFilename

* fix tests?

* update torrent and file size, some more fixes/improvements

* use higher bitrate for high fps video, adjust bitrates

* add test video

* don't throw error if resolution is undefined

* generate test fixture on the fly

* use random noise video for bitrate test, add promise

* shorten test video to avoid timeout

* use existing function to optimize video

* various fixes

* increase test timeout

* limit test fixture size, add link

* test fixes

* add await

* more test fixes, add -b:v parameter

* replace ffmpeg wiki link

* fix ffmpeg params

* fix unit test

* add test fixture to .gitgnore

* add video transcoding fps model

* add missing file
2018-10-08 16:26:04 +02:00
Chocobozzz 499d901595
Fix config endpoint 2018-10-03 14:36:00 +02:00
Rigel Kent abb2c7927c show last commit hash alongside server version in footer 2018-10-01 14:31:01 +02:00
Chocobozzz 7348b1fd84
Speed up overviews route 2018-09-14 11:52:23 +02:00
Chocobozzz 62689b942b
Correctly migrate to fs-extra 2018-08-27 16:23:34 +02:00
Chocobozzz 06215f15e0
Cleanup utils helper 2018-08-14 15:28:30 +02:00
Chocobozzz 3e17515e29 Add torrent tests 2018-08-08 09:30:31 +02:00
Chocobozzz 990b6a0b0c Import torrents with webtorrent 2018-08-08 09:30:31 +02:00
Chocobozzz ce33919c24 Import magnets with webtorrent 2018-08-08 09:30:31 +02:00
Chocobozzz cf7a61b5a2
Cleanup req files on bad request 2018-07-31 15:09:34 +02:00
Chocobozzz 2cebd79701
Fix last commit 2018-07-26 11:02:04 +02:00
Rigel Kent c1e791bad0
expliciting type checks and predicates (server only) 2018-07-25 22:01:25 +02:00
Chocobozzz 98d3324db3
Fix nodeinfo endpoint 2018-07-24 14:36:25 +02:00
Rigel Kent 3f6d68d967 adding initial support for nodeinfo 2018-07-24 14:08:44 +02:00
Chocobozzz 2186386cca
Add concept of video state, and add ability to wait transcoding before
publishing a video
2018-06-12 20:37:51 +02:00
Rigel Kent ff2c1fe813 feature: IP filtering on signup page
disable registration form on IP not in range
checking the CIDR list before filtering with it
placing the cidr filters as an attribute object in the config
2018-05-22 19:44:34 +02:00
Chocobozzz 0626e7af82
Add account view 2018-04-24 15:13:19 +02:00
Chocobozzz d5b7d9110d
Fix error logging 2018-03-26 16:04:14 +02:00
Chocobozzz ac81d1a06d
Add ability to set video thumbnail/preview 2018-02-14 16:03:09 +01:00
Chocobozzz fd206f0b2d
Add ability to update some configuration keys 2018-01-17 10:41:27 +01:00
Chocobozzz 47564bbe2e
Add ability to disable video comments 2018-01-03 10:38:19 +01:00
Chocobozzz c5911fd347
Begin to add avatar to actors 2017-12-29 19:10:13 +01:00
Chocobozzz 0405ab52dc
Set port if not specified in webfinger 2017-12-19 14:21:14 +01:00
Chocobozzz 50d6de9c28
Begin moving video channel to actor 2017-12-19 10:53:16 +01:00
Chocobozzz 3fd3ab2d34
Move models to typescript-sequelize 2017-12-13 16:50:33 +01:00
Chocobozzz 8d468a16fd
Cleanup helpers 2017-11-27 19:40:53 +01:00
Chocobozzz efc32059d9
Send server announce when users upload a video 2017-11-27 19:40:52 +01:00
Chocobozzz 7a7724e66e
Handle follow/accept 2017-11-27 19:40:51 +01:00
Chocobozzz 404b54e14f
Adapt client with video channels 2017-10-26 09:11:38 +02:00
Chocobozzz f5028693a8
Use async/await in lib and initializers 2017-10-26 09:11:38 +02:00
Chocobozzz eb08047657
Use async/await in controllers 2017-10-26 09:11:38 +02:00
Chocobozzz 14d3270f36
Change how we handle resolution
It was an enum before, now we just use video height
2017-10-09 11:17:36 +02:00
Chocobozzz 40298b0254 Implement video transcoding on server side 2017-10-03 15:31:26 +02:00
Green-Star 792dbaf07f Handle blacklist (#84)
* Client: Add list blacklist feature

* Server: Add list blacklist feature

* Client: Add videoId column

* Server: Add some video infos in the REST api

* Client: Add video information in the blacklist list

* Fix sortable columns :)

* Client: Add removeFromBlacklist feature

* Server: Add removeFromBlacklist feature

* Move to TypeScript

* Move to TypeScript and Promises

* Server: Fix blacklist list sort

* Server: Fetch videos informations

* Use common shared interface for client and server

* Add check-params remove blacklisted video tests

* Add check-params list blacklisted videos tests

* Add list blacklist tests

* Add remove from blacklist tests

* Add video blacklist management tests

* Fix rebase onto develop issues

* Server: Add sort on blacklist id column

* Server: Add blacklists library

* Add blacklist id sort test

* Add check-params tests for blacklist list pagination, count and sort

* Fix coding style

* Increase Remote API tests timeout

* Increase Request scheduler API tests timeout

* Fix typo

* Increase video transcoding API tests timeout

* Move tests to Typescript

* Use lodash orderBy method

* Fix typos

* Client: Remove optional tests in blacklist model attributes

* Move blacklist routes from 'blacklists' to 'blacklist'

* CLient: Remove blacklist-list.component.scss

* Rename 'blacklists' files to 'blacklist'

* Use only BlacklistedVideo interface

* Server: Use getFormattedObjects method in listBlacklist method

* Client: Use new coding style

* Server: Use new sort validator methods

* Server: Use new checkParams methods

* Client: Fix sortable columns
2017-09-22 09:13:43 +02:00
Chocobozzz 0aef76c479 Formated -> Formatted 2017-08-25 11:45:31 +02:00
Chocobozzz 291e8d3eed Add ability to limit user registrations 2017-07-25 20:17:28 +02:00