feat: add comprehensive NodeJS development skills and rules

Based on Planner and Memory Manager analysis:

New Skills (8):
- nodejs-express-patterns: App structure, routing, middleware
- nodejs-security-owasp: OWASP Top 10 security practices
- nodejs-testing-jest: Unit/integration tests, mocking
- nodejs-auth-jwt: JWT authentication, OAuth, sessions
- nodejs-error-handling: Error classes, middleware, async handlers
- nodejs-middleware-patterns: Auth, validation, rate limiting
- nodejs-db-patterns: SQLite, PostgreSQL, MongoDB patterns
- nodejs-npm-management: package.json, scripts, dependencies

New Rules:
- nodejs.md: Code style, security, best practices

Updated:
- backend-developer.md: Added skills reference table

Milestone: #48 NodeJS Development Coverage
Related: Planner & Memory Manager analysis results
This commit is contained in:
¨NW¨
2026-04-05 02:39:06 +01:00
parent fbc1f6122f
commit 8fcd8f8a9b
10 changed files with 3318 additions and 1 deletions

View File

@@ -246,6 +246,39 @@ module.exports = errorHandler;
- DO NOT use synchronous operations in request handlers
- DO NOT hardcode secrets or credentials
## Skills Reference
This agent uses the following skills for comprehensive Node.js development:
### Core Skills
| Skill | Purpose |
|-------|---------|
| `nodejs-express-patterns` | Express app structure, routing, middleware |
| `nodejs-error-handling` | Error classes, middleware, async handlers |
| `nodejs-middleware-patterns` | Authentication, validation, rate limiting |
| `nodejs-auth-jwt` | JWT authentication, OAuth, sessions |
| `nodejs-security-owasp` | OWASP Top 10, security best practices |
### Testing & Quality
| Skill | Purpose |
|-------|---------|
| `nodejs-testing-jest` | Unit tests, integration tests, mocking |
### Database
| Skill | Purpose |
|-------|---------|
| `nodejs-db-patterns` | SQLite, PostgreSQL, MongoDB patterns |
### Package Management
| Skill | Purpose |
|-------|---------|
| `nodejs-npm-management` | package.json, scripts, dependencies |
### Rules
| File | Content |
|------|---------|
| `.kilo/rules/nodejs.md` | Code style, security, best practices |
## Handoff Protocol
After implementation:
@@ -253,7 +286,8 @@ After implementation:
2. Check security headers
3. Test error handling
4. Create database migration
5. Tag `@CodeSkeptic` for review
5. Run tests with `npm test`
6. Tag `@CodeSkeptic` for review
## Gitea Commenting (MANDATORY)
**You MUST post a comment to the Gitea issue after completing your work.**

271
.kilo/rules/nodejs.md Normal file
View File

@@ -0,0 +1,271 @@
# NodeJS Development Rules
Essential rules for Node.js backend development.
## Code Style
- Use `const` and `let`, never `var`
- Use arrow functions for callbacks
- Use async/await instead of callbacks
- Use template literals for string interpolation
- Use object destructuring
- Use spread operator for objects/arrays
```javascript
// ✅ Good
const { id, name } = req.body;
const user = { ...req.body, createdAt: new Date() };
const users = await User.findAll();
// ❌ Bad
var id = req.body.id;
const user = Object.assign({}, req.body, { createdAt: new Date() });
User.findAll().then(users => {});
```
## Error Handling
- Always use try/catch with async/await
- Use centralized error handling middleware
- Never catch and swallow errors
- Use custom AppError classes
- Log errors with context
```javascript
// ✅ Good
try {
const user = await User.findById(id);
if (!user) throw new NotFoundError('User');
res.json({ user });
} catch (error) {
next(error);
}
// ❌ Bad
User.findById(id).then(user => {
if (!user) return res.status(404).json({ error: 'Not found' });
res.json({ user });
}).catch(err => {}); // Swallowing error
```
## Async Code
- Always use async/await
- Never mix callbacks and promises
- Use Promise.all for parallel operations
- Use async middleware wrapper
```javascript
// ✅ Good
const [users, posts] = await Promise.all([
User.findAll(),
Post.findAll()
]);
// ❌ Bad
let users;
User.findAll().then(u => { users = u; });
console.log(users); // undefined
```
## Security
- Always validate and sanitize input
- Use parameterized queries
- Never expose sensitive data
- Use HTTPS in production
- Set security headers with helmet
- Rate limit public endpoints
```javascript
// ✅ Good
const user = await db.query('SELECT * FROM users WHERE id = ?', [id]);
app.use(helmet());
// ❌ Bad
const user = await db.query(`SELECT * FROM users WHERE id = ${id}`);
// SQL injection vulnerable
```
## Authentication
- Never store passwords in plain text
- Use bcrypt for password hashing
- Use short-lived access tokens
- Use refresh tokens
- Use httpOnly cookies
- Never put secrets in JWT payload
```javascript
// ✅ Good
const hashedPassword = await bcrypt.hash(password, 12);
const token = jwt.sign({ id: user.id }, process.env.JWT_SECRET, { expiresIn: '1h' });
// ❌ Bad
const hashedPassword = password; // No hash
const token = jwt.sign({ password: user.password }, 'secret'); // Secret in payload
```
## Express Best Practices
- Use express.Router() for route organization
- Keep route handlers thin
- Validate at route level
- Put error handlers last
- Use middleware for cross-cutting concerns
```javascript
// ✅ Good
// routes/users.js
const router = express.Router();
router.get('/', authenticate, validate, controller.list);
// app.js
app.use('/api/users', routes.users);
app.use(errorHandler); // Last middleware
// ❌ Bad
app.get('/api/users', async (req, res) => {
// All logic in route
});
```
## Database
- Use connection pooling
- Close connections gracefully
- Use transactions for writes
- Index frequently queried fields
- Use migrations for schema changes
```javascript
// ✅ Good
await db.transaction(async (trx) => {
await trx('users').insert(user);
await trx('profiles').insert(profile);
});
// ❌ Bad
async function createUser(data) {
const user = await db('users').insert(data);
// No transaction, partial data on error
await Profile.create({ userId: user.id });
}
```
## Logging
- Use structured logging (pino, winston)
- Log levels: error, warn, info, debug
- Include request ID for tracing
- Log errors with stack traces
- Don't log sensitive data
```javascript
// ✅ Good
logger.info({ userId, action: 'login', ip: req.ip });
// ❌ Bad
console.log('User logged in:', user); // Logs entire user including password
```
## Testing
- Write tests for critical paths
- Use Jest or Mocha
- Mock external dependencies
- Aim for 80%+ coverage
- Test edge cases
```javascript
// ✅ Good
describe('UserService', () => {
it('should create user with hashed password', async () => {
const user = await service.create({ email, password });
expect(user.password).not.toBe(password);
});
});
```
## Environment
- Use .env for secrets
- Never commit secrets
- Use different configs for environments
- Validate required env vars
```javascript
// ✅ Good
const config = {
db: {
url: process.env.DATABASE_URL
}
};
if (!config.db.url) {
throw new Error('DATABASE_URL is required');
}
// ❌ Bad
const config = {
db: {
url: 'postgres://user:pass@localhost/db' // Hardcoded
}
};
```
## Package Management
- Use exact versions in production
- Run npm audit regularly
- Update dependencies regularly
- Remove unused dependencies
```bash
# ✅ Good
npm audit
npx depcheck
# ❌ Bad
# Never running security audit
# Many unused dependencies
```
## Performance
- Use streaming for large files
- Cache frequently accessed data
- Use connection pooling
- Implement pagination
- Compress responses
```javascript
// ✅ Good
app.use(compression());
app.get('/users', paginated, controller.list);
// ❌ Bad
app.get('/users', async (req, res) => {
const users = await User.findAll(); // All users at once
res.json(users);
});
```
## Clean Code
- No magic numbers, use constants
- Meaningful variable names
- One function, one responsibility
- Comments only for "why", not "what"
- DRY principle
```javascript
// ✅ Good
const MAX_LOGIN_ATTEMPTS = 5;
const isLocked = user.loginAttempts >= MAX_LOGIN_ATTEMPTS;
// ❌ Bad
if (user.loginAttempts >= 5) { // Magic number
// ...
}
```

View File

@@ -0,0 +1,402 @@
# NodeJS Authentication with JWT
Comprehensive authentication patterns using JSON Web Tokens.
## JWT Setup
```bash
npm install jsonwebtoken bcrypt dotenv
```
## Token Generation
```javascript
// src/utils/jwt.js
const jwt = require('jsonwebtoken');
const JWT_SECRET = process.env.JWT_SECRET;
const JWT_EXPIRES_IN = process.env.JWT_EXPIRES_IN || '7d';
const JWT_REFRESH_EXPIRES_IN = '30d';
function generateToken(payload) {
return jwt.sign(payload, JWT_SECRET, { expiresIn: JWT_EXPIRES_IN });
}
function generateRefreshToken(payload) {
return jwt.sign(payload, JWT_SECRET, { expiresIn: JWT_REFRESH_EXPIRES_IN });
}
function verifyToken(token) {
try {
return jwt.verify(token, JWT_SECRET);
} catch (error) {
if (error.name === 'TokenExpiredError') {
throw new AppError('Token expired', 401);
}
throw new AppError('Invalid token', 401);
}
}
module.exports = { generateToken, generateRefreshToken, verifyToken };
```
## Password Hashing
```javascript
// src/utils/password.js
const bcrypt = require('bcrypt');
const SALT_ROUNDS = 12;
async function hashPassword(password) {
return bcrypt.hash(password, SALT_ROUNDS);
}
async function comparePassword(password, hash) {
return bcrypt.compare(password, hash);
}
module.exports = { hashPassword, comparePassword };
```
## Authentication Middleware
```javascript
// src/middleware/auth.js
const { verifyToken } = require('../utils/jwt');
const AppError = require('../utils/AppError');
function authenticate(req, res, next) {
// Get token from header
const authHeader = req.headers.authorization;
if (!authHeader || !authHeader.startsWith('Bearer ')) {
return next(new AppError('Authentication required', 401));
}
const token = authHeader.split(' ')[1];
try {
const decoded = verifyToken(token);
req.user = decoded;
next();
} catch (error) {
next(error);
}
}
function authorize(...roles) {
return (req, res, next) => {
if (!roles.includes(req.user.role)) {
return next(new AppError('Insufficient permissions', 403));
}
next();
};
}
module.exports = { authenticate, authorize };
```
## Auth Service
```javascript
// src/services/authService.js
const User = require('../models/User');
const { hashPassword, comparePassword } = require('../utils/password');
const { generateToken, generateRefreshToken } = require('../utils/jwt');
const AppError = require('../utils/AppError');
class AuthService {
async register(userData) {
// Check if user exists
const existingUser = await User.findByEmail(userData.email);
if (existingUser) {
throw new AppError('Email already registered', 409);
}
// Hash password
const hashedPassword = await hashPassword(userData.password);
// Create user
const user = await User.create({
...userData,
password: hashedPassword,
role: 'user'
});
// Generate tokens
const accessToken = generateToken({ id: user.id, role: user.role });
const refreshToken = generateRefreshToken({ id: user.id });
return { user: user.toSafeObject(), accessToken, refreshToken };
}
async login(email, password) {
// Find user
const user = await User.findByEmail(email);
if (!user) {
throw new AppError('Invalid credentials', 401);
}
// Compare password
const isValid = await comparePassword(password, user.password);
if (!isValid) {
throw new AppError('Invalid credentials', 401);
}
// Generate tokens
const accessToken = generateToken({ id: user.id, role: user.role });
const refreshToken = generateRefreshToken({ id: user.id });
return { user: user.toSafeObject(), accessToken, refreshToken };
}
async refresh(refreshToken) {
const decoded = verifyToken(refreshToken);
const user = await User.findById(decoded.id);
if (!user) {
throw new AppError('User not found', 404);
}
const newAccessToken = generateToken({ id: user.id, role: user.role });
return { accessToken: newAccessToken };
}
async logout(userId) {
// Invalidate refresh token (store in blacklist or remove from db)
await User.updateRefreshToken(userId, null);
}
}
module.exports = new AuthService();
```
## Auth Controller
```javascript
// src/controllers/authController.js
const authService = require('../services/authService');
const cookieOptions = {
httpOnly: true,
secure: process.env.NODE_ENV === 'production',
sameSite: 'strict',
maxAge: 7 * 24 * 60 * 60 * 1000 // 7 days
};
class AuthController {
async register(req, res, next) {
try {
const { user, accessToken, refreshToken } = await authService.register(req.body);
res.cookie('refreshToken', refreshToken, cookieOptions);
res.status(201).json({
status: 'success',
data: { user, accessToken }
});
} catch (error) {
next(error);
}
}
async login(req, res, next) {
try {
const { email, password } = req.body;
const { user, accessToken, refreshToken } = await authService.login(email, password);
res.cookie('refreshToken', refreshToken, cookieOptions);
res.json({
status: 'success',
data: { user, accessToken }
});
} catch (error) {
next(error);
}
}
async refresh(req, res, next) {
try {
const refreshToken = req.cookies?.refreshToken;
if (!refreshToken) {
throw new AppError('Refresh token required', 401);
}
const { accessToken } = await authService.refresh(refreshToken);
res.json({
status: 'success',
data: { accessToken }
});
} catch (error) {
next(error);
}
}
async logout(req, res, next) {
try {
await authService.logout(req.user.id);
res.clearCookie('refreshToken');
res.json({ status: 'success', message: 'Logged out' });
} catch (error) {
next(error);
}
}
async me(req, res, next) {
try {
res.json({
status: 'success',
data: { user: req.user }
});
} catch (error) {
next(error);
}
}
}
module.exports = new AuthController();
```
## Routes
```javascript
// src/routes/auth.js
const router = require('express').Router();
const authController = require('../controllers/authController');
const { authenticate } = require('../middleware/auth');
const { validateUser } = require('../middleware/validation');
// Public routes
router.post('/register', validateUser, authController.register);
router.post('/login', authController.login);
router.post('/refresh', authController.refresh);
router.post('/logout', authController.logout);
// Protected routes
router.get('/me', authenticate, authController.me);
module.exports = router;
```
## OAuth Integration
```javascript
// src/services/oauthService.js
const { generateToken } = require('../utils/jwt');
const User = require('../models/User');
class OAuthService {
async googleAuth(code) {
// Exchange code for tokens
const tokens = await this.exchangeGoogleCode(code);
// Get user info
const userInfo = await this.getGoogleUserInfo(tokens.access_token);
// Find or create user
let user = await User.findByEmail(userInfo.email);
if (!user) {
user = await User.create({
email: userInfo.email,
name: userInfo.name,
provider: 'google',
providerId: userInfo.id,
verified: true
});
}
// Generate JWT
const accessToken = generateToken({ id: user.id, role: user.role });
return { user: user.toSafeObject(), accessToken };
}
async exchangeGoogleCode(code) {
const response = await fetch('https://oauth2.googleapis.com/token', {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({
code,
client_id: process.env.GOOGLE_CLIENT_ID,
client_secret: process.env.GOOGLE_CLIENT_SECRET,
redirect_uri: process.env.GOOGLE_REDIRECT_URI,
grant_type: 'authorization_code'
})
});
return response.json();
}
async getGoogleUserInfo(accessToken) {
const response = await fetch('https://www.googleapis.com/oauth2/v2/userinfo', {
headers: { Authorization: `Bearer ${accessToken}` }
});
return response.json();
}
}
module.exports = new OAuthService();
```
## Security Best Practices
1. **Use HTTPS** - Never send tokens over HTTP
2. **Short expiration** - Access tokens: 15min - 1hr
3. **Refresh tokens** - Long-lived, stored securely
4. **HttpOnly cookies** - For refresh tokens
5. **Token blacklist** - For logout/compromised tokens
6. **Rate limiting** - On auth endpoints
7. **Strong secrets** - 256+ bits for JWT_SECRET
8. **Don't store sensitive data** - In JWT payload
## Testing Auth
```javascript
// Tests for authentication
const request = require('supertest');
const app = require('../src/app');
describe('Auth', () => {
it('should register new user', async () => {
const res = await request(app)
.post('/auth/register')
.send({ email: 'test@test.com', password: 'Password123!' })
.expect(201);
expect(res.body.data.accessToken).toBeDefined();
});
it('should login existing user', async () => {
// Create user first
await request(app)
.post('/auth/register')
.send({ email: 'test@test.com', password: 'Password123!' });
// Login
const res = await request(app)
.post('/auth/login')
.send({ email: 'test@test.com', password: 'Password123!' })
.expect(200);
expect(res.body.data.accessToken).toBeDefined();
});
it('should access protected route', async () => {
// Get token
const token = await getTestToken();
const res = await request(app)
.get('/auth/me')
.set('Authorization', `Bearer ${token}`)
.expect(200);
expect(res.body.data.user).toBeDefined();
});
});
```

View File

@@ -0,0 +1,433 @@
# NodeJS Database Patterns
Database patterns for SQLite, PostgreSQL, and MongoDB.
## SQLite with Knex
### Connection
```javascript
// src/db/connection.js
const knex = require('knex');
const db = knex({
client: 'better-sqlite3',
connection: {
filename: process.env.DATABASE_URL || './data.db'
},
useNullAsDefault: true,
pool: {
min: 2,
max: 10
}
});
module.exports = db;
```
### Migrations
```javascript
// migrations/20240101000000_create_users.js
exports.up = function(knex) {
return knex.schema.createTable('users', table => {
table.increments('id').primary();
table.string('email').unique().notNullable();
table.string('password').notNullable();
table.string('name');
table.enu('role', ['user', 'admin']).defaultTo('user');
table.timestamp('created_at').defaultTo(knex.fn.now());
table.timestamp('updated_at').defaultTo(knex.fn.now());
table.index('email');
});
};
exports.down = function(knex) {
return knex.schema.dropTable('users');
};
```
### Model
```javascript
// src/models/User.js
const db = require('../db/connection');
class User {
static async create(data) {
const [id] = await db('users').insert(data);
return this.findById(id);
}
static async findById(id) {
return db('users').where({ id }).first();
}
static async findByEmail(email) {
return db('users').where({ email }).first();
}
static async findAll(options = {}) {
const { page = 1, limit = 20 } = options;
const offset = (page - 1) * limit;
return db('users')
.select('id', 'email', 'name', 'role', 'created_at')
.limit(limit)
.offset(offset);
}
static async update(id, data) {
await db('users').where({ id }).update({
...data,
updated_at: db.fn.now()
});
return this.findById(id);
}
static async delete(id) {
return db('users').where({ id }).del();
}
toSafeObject() {
const { password, ...safe } = this;
return safe;
}
}
module.exports = User;
```
### Transactions
```javascript
const db = require('./db/connection');
async function transferFunds(fromId, toId, amount) {
return db.transaction(async (trx) => {
// Check balance
const from = await trx('accounts').where({ id: fromId }).first();
if (from.balance < amount) {
throw new Error('Insufficient funds');
}
// Debit
await trx('accounts')
.where({ id: fromId })
.decrement('balance', amount);
// Credit
await trx('accounts')
.where({ id: toId })
.increment('balance', amount);
// Log transaction
await trx('transactions').insert({
from_id: fromId,
to_id: toId,
amount
});
});
}
```
## PostgreSQL
### Connection (pg)
```javascript
const { Pool } = require('pg');
const pool = new Pool({
host: process.env.PG_HOST,
port: process.env.PG_PORT,
database: process.env.PG_DATABASE,
user: process.env.PG_USER,
password: process.env.PG_PASSWORD,
max: 20,
idleTimeoutMillis: 30000
});
// Query helper
async function query(text, params) {
const start = Date.now();
const result = await pool.query(text, params);
const duration = Date.now() - start;
console.log('Query:', { text: text.substring(0, 50), duration, rows: result.rowCount });
return result;
}
module.exports = { query, pool };
```
### Prepared Statements
```javascript
// Named parameters
const { query } = require('./db');
const sql = 'SELECT * FROM users WHERE email = $1 AND role = $2';
const result = await query(sql, ['user@example.com', 'admin']);
// With parameterized queries
const text = 'INSERT INTO users (email, name) VALUES ($1, $2) RETURNING *';
const values = ['user@example.com', 'John'];
const result = await query(text, values);
```
### Connection Pooling
```javascript
const { Pool } = require('pg');
const pool = new Pool({
max: 20,
idleTimeoutMillis: 30000,
connectionTimeoutMillis: 2000
});
pool.on('error', (err) => {
console.error('Unexpected error on idle client', err);
});
// In-request connection
app.get('/users/:id', async (req, res) => {
const client = await pool.connect();
try {
const result = await client.query('SELECT * FROM users WHERE id = $1', [req.params.id]);
res.json(result.rows[0]);
} finally {
client.release();
}
});
```
## MongoDB
### Connection (Mongoose)
```javascript
const mongoose = require('mongoose');
mongoose.connect(process.env.MONGODB_URI, {
useNewUrlParser: true,
useUnifiedTopology: true
});
mongoose.connection.on('connected', () => {
console.log('MongoDB connected');
});
mongoose.connection.on('error', (err) => {
console.error('MongoDB error:', err);
});
```
### Schema
```javascript
// src/models/User.js
const mongoose = require('mongoose');
const userSchema = new mongoose.Schema({
email: {
type: String,
required: true,
unique: true,
lowercase: true,
trim: true
},
password: {
type: String,
required: true
},
name: {
type: String,
trim: true,
maxlength: 100
},
role: {
type: String,
enum: ['user', 'admin'],
default: 'user'
}
}, {
timestamps: true
});
// Indexes
userSchema.index({ email: 1 });
// Methods
userSchema.methods.toSafeObject = function() {
const { password, ...safe } = this.toObject();
return safe;
};
// Statics
userSchema.statics.findByEmail = function(email) {
return this.findOne({ email });
};
module.exports = mongoose.model('User', userSchema);
```
### CRUD Operations
```javascript
// Create
const user = await User.create({
email: 'test@example.com',
password: hashedPassword,
name: 'Test User'
});
// Read
const user = await User.findById(id);
const user = await User.findOne({ email });
const users = await User.find({ role: 'user' }).limit(20);
// Update
const user = await User.findByIdAndUpdate(id, data, { new: true });
const result = await User.updateMany({ role: 'user' }, { $set: { active: true } });
// Delete
await User.findByIdAndDelete(id);
await User.deleteMany({ inactive: true });
```
### Aggregation
```javascript
const stats = await User.aggregate([
{ $match: { role: 'user' } },
{ $group: {
_id: '$role',
total: { $sum: 1 },
avgPosts: { $avg: '$postCount' }
}},
{ $sort: { total: -1 } },
{ $limit: 10 }
]);
```
### Pagination
```javascript
async function paginate(Model, filter, options = {}) {
const { page = 1, limit = 20 } = options;
const skip = (page - 1) * limit;
const [docs, total] = await Promise.all([
Model.find(filter).skip(skip).limit(limit),
Model.countDocuments(filter)
]);
return {
docs,
total,
page,
pages: Math.ceil(total / limit)
};
}
```
## Common Patterns
### Repository Pattern
```javascript
// src/repositories/BaseRepository.js
class BaseRepository {
constructor(model) {
this.model = model;
}
async findById(id) {
return this.model.findById(id);
}
async findOne(filter) {
return this.model.findOne(filter);
}
async findAll(filter, options = {}) {
const { page, limit } = options;
let query = this.model.find(filter);
if (page && limit) {
query = query.skip((page - 1) * limit).limit(limit);
}
return query;
}
async create(data) {
const doc = new this.model(data);
return doc.save();
}
async update(id, data) {
return this.model.findByIdAndUpdate(id, data, { new: true });
}
async delete(id) {
return this.model.findByIdAndDelete(id);
}
}
module.exports = BaseRepository;
```
### Query Builder
```javascript
// src/utils/queryBuilder.js
class QueryBuilder {
constructor(model) {
this.query = model.find();
}
filter(filters) {
if (filters) {
this.query = this.query.where(filters);
}
return this;
}
search(field, term) {
if (term) {
this.query = this.query.where({ [field]: new RegExp(term, 'i') });
}
return this;
}
sort(field, direction = 'asc') {
if (field) {
this.query = this.query.sort({ [field]: direction });
}
return this;
}
paginate(page = 1, limit = 20) {
const skip = (page - 1) * limit;
this.query = this.query.skip(skip).limit(limit);
return this;
}
build() {
return this.query;
}
}
// Usage
const users = await new QueryBuilder(User)
.filter({ role: 'user' })
.search('name', 'john')
.sort('createdAt', 'desc')
.paginate(1, 20)
.build();
```

View File

@@ -0,0 +1,341 @@
# NodeJS Error Handling
Comprehensive error handling patterns for Node.js applications.
## Error Classes
```javascript
// src/utils/AppError.js
class AppError extends Error {
constructor(message, statusCode) {
super(message);
this.statusCode = statusCode;
this.status = `${statusCode}`.startsWith('4') ? 'fail' : 'error';
this.isOperational = true;
Error.captureStackTrace(this, this.constructor);
}
}
class NotFoundError extends AppError {
constructor(resource = 'Resource') {
super(`${resource} not found`, 404);
}
}
class ValidationError extends AppError {
constructor(errors) {
super('Validation failed', 400);
this.errors = errors;
}
}
class UnauthorizedError extends AppError {
constructor(message = 'Unauthorized') {
super(message, 401);
}
}
class ForbiddenError extends AppError {
constructor(message = 'Forbidden') {
super(message, 403);
}
}
class ConflictError extends AppError {
constructor(message = 'Conflict') {
super(message, 409);
}
}
module.exports = {
AppError,
NotFoundError,
ValidationError,
UnauthorizedError,
ForbiddenError,
ConflictError
};
```
## Error Middleware
```javascript
// src/middleware/error.js
const AppError = require('../utils/AppError');
// Not found handler
function notFoundHandler(req, res, next) {
next(new AppError(`Cannot find ${req.originalUrl} on this server`, 404));
}
// Global error handler
function errorHandler(err, req, res, next) {
err.statusCode = err.statusCode || 500;
err.status = err.status || 'error';
if (process.env.NODE_ENV === 'development') {
sendErrorDev(err, res);
} else {
sendErrorProd(err, res);
}
}
function sendErrorDev(err, res) {
res.status(err.statusCode).json({
status: err.status,
message: err.message,
stack: err.stack,
error: err
});
}
function sendErrorProd(err, res) {
// Operational error: send message to client
if (err.isOperational) {
res.status(err.statusCode).json({
status: err.status,
message: err.message
});
} else {
// Programming error: don't leak details
console.error('ERROR:', err);
res.status(500).json({
status: 'error',
message: 'Something went wrong'
});
}
}
module.exports = { notFoundHandler, errorHandler };
```
## Async Error Wrapper
```javascript
// src/middleware/asyncHandler.js
function asyncHandler(fn) {
return (req, res, next) => {
Promise.resolve(fn(req, res, next)).catch(next);
};
}
// Usage
router.get('/users/:id', asyncHandler(controller.getById));
// Or wrap entire controller
const wrapController = (controller) => {
const wrapped = {};
for (const [key, fn] of Object.entries(controller)) {
wrapped[key] = asyncHandler(fn.bind(controller));
}
return wrapped;
};
module.exports = { asyncHandler, wrapController };
```
## Validation Errors
```javascript
// src/middleware/validation.js
const { validationResult } = require('express-validator');
const { ValidationError } = require('../utils/AppError');
function validate(validations) {
return async (req, res, next) => {
await Promise.all(validations.map(validation => validation.run(req)));
const errors = validationResult(req);
if (errors.isEmpty()) {
return next();
}
const extractedErrors = errors.array().map(err => ({
field: err.path,
message: err.msg,
value: err.value
}));
return next(new ValidationError(extractedErrors));
};
}
module.exports = { validate };
```
## Database Error Handling
```javascript
// src/middleware/dbErrorHandler.js
const AppError = require('../utils/AppError');
function dbErrorHandler(err, req, res, next) {
// PostgreSQL unique violation
if (err.code === '23505') {
const field = err.constraint.split('_')[0];
return next(new AppError(`${field} already exists`, 409));
}
// PostgreSQL foreign key violation
if (err.code === '23503') {
return next(new AppError('Referenced resource not found', 400));
}
// SQLite unique violation
if (err.code === 'SQLITE_CONSTRAINT_UNIQUE') {
return next(new AppError('Resource already exists', 409));
}
// MongoDB duplicate key
if (err.code === 11000) {
const field = Object.keys(err.keyValue)[0];
return next(new AppError(`${field} already exists`, 409));
}
// MongoDB validation error
if (err.name === 'ValidationError') {
const messages = Object.values(err.errors).map(e => e.message);
return next(new AppError(messages.join('. '), 400));
}
// MongoDB cast error
if (err.name === 'CastError') {
return next(new AppError('Invalid ID format', 400));
}
next(err);
}
module.exports = dbErrorHandler;
```
## Logging Errors
```javascript
// src/utils/logger.js
const pino = require('pino');
const logger = pino({
level: process.env.LOG_LEVEL || 'info',
transport: process.env.NODE_ENV === 'development'
? { target: 'pino-pretty' }
: undefined
});
function logError(err, req = null) {
logger.error({
message: err.message,
stack: err.stack,
statusCode: err.statusCode || 500,
path: req?.path,
method: req?.method,
user: req?.user?.id,
body: req?.body,
query: req?.query,
params: req?.params,
timestamp: new Date().toISOString()
});
}
function logInfo(message, data = {}) {
logger.info({ message, ...data });
}
module.exports = { logger, logError, logInfo };
```
## Unhandled Rejections
```javascript
// src/server.js
process.on('unhandledRejection', (err) => {
console.error('UNHANDLED REJECTION:', err);
logError(err);
// Graceful shutdown
server.close(() => {
process.exit(1);
});
});
process.on('uncaughtException', (err) => {
console.error('UNCAUGHT EXCEPTION:', err);
logError(err);
// Must exit immediately for uncaught exceptions
process.exit(1);
});
```
## API Error Responses
```javascript
// Standardized error responses
const errorResponses = {
400: {
status: 'fail',
message: 'Bad Request',
errors: []
},
401: {
status: 'fail',
message: 'Unauthorized'
},
403: {
status: 'fail',
message: 'Forbidden'
},
404: {
status: 'fail',
message: 'Not Found'
},
409: {
status: 'fail',
message: 'Conflict'
},
500: {
status: 'error',
message: 'Internal Server Error'
}
};
// Example responses
{
"status": "fail",
"message": "Validation failed",
"errors": [
{ "field": "email", "message": "Invalid email format" },
{ "field": "password", "message": "Must be at least 8 characters" }
]
}
```
## Error Handling in Tests
```javascript
// tests/unit/errors.test.js
describe('Error Handling', () => {
it('should handle AppError', () => {
const error = new AppError('Test error', 400);
expect(error.statusCode).toBe(400);
expect(error.status).toBe('fail');
expect(error.isOperational).toBe(true);
});
it('should handle async errors', async () => {
const req = {};
const res = {};
const next = jest.fn();
const handler = asyncHandler(async () => {
throw new AppError('Test', 400);
});
await handler(req, res, next);
expect(next).toHaveBeenCalledWith(expect.any(AppError));
});
});
```

View File

@@ -0,0 +1,334 @@
# NodeJS Express Patterns
Comprehensive patterns for building production-ready Express.js applications.
## Overview
This skill provides canonical patterns for Express.js development including routing, middleware, error handling, and best practices.
## Express Application Structure
```
backend/
├── src/
│ ├── app.js # Express app setup
│ ├── server.js # Server entry point
│ ├── config/
│ │ ├── index.js # Config aggregation
│ │ ├── database.js # DB config
│ │ └── security.js # Security config
│ ├── routes/
│ │ ├── index.js # Route aggregator
│ │ ├── api/ # Public API routes
│ │ │ ├── users.js
│ │ │ ├── posts.js
│ │ │ └── index.js
│ │ └── admin/ # Admin API routes
│ │ ├── auth.js
│ │ ├── users.js
│ │ └── index.js
│ ├── controllers/ # Route handlers
│ ├── services/ # Business logic
│ ├── models/ # Data models
│ ├── middleware/
│ │ ├── auth.js
│ │ ├── validation.js
│ │ ├── error.js
│ │ └── rateLimit.js
│ ├── utils/
│ └── db/
│ ├── connection.js
│ └── migrations/
├── tests/
├── package.json
└── docker-compose.yml
```
## Core Patterns
### 1. App Bootstrap (app.js)
```javascript
// backend/src/app.js
const express = require('express');
const helmet = require('helmet');
const cors = require('cors');
const rateLimit = require('express-rate-limit');
const { errorHandler, notFoundHandler } = require('./middleware/error');
const routes = require('./routes');
const app = express();
// Security middleware
app.use(helmet());
app.use(cors({
origin: process.env.CORS_ORIGIN?.split(',') || '*',
credentials: true
}));
// Rate limiting
const limiter = rateLimit({
windowMs: 15 * 60 * 1000, // 15 minutes
max: 100, // limit each IP to 100 requests per windowMs
standardHeaders: true,
legacyHeaders: false
});
app.use('/api/', limiter);
// Body parsing
app.use(express.json({ limit: '10kb' }));
app.use(express.urlencoded({ extended: true, limit: '10kb' }));
// Static files
app.use('/media', express.static(path.join(__dirname, '../uploads')));
// Routes
app.use('/api', routes.api);
app.use('/api/admin', routes.admin);
// Health check
app.get('/health', (req, res) => {
res.json({ status: 'ok', timestamp: new Date().toISOString() });
});
// Error handlers (must be last)
app.use(notFoundHandler);
app.use(errorHandler);
module.exports = app;
```
### 2. Server Entry Point (server.js)
```javascript
// backend/src/server.js
const app = require('./app');
const { connectDB } = require('./db/connection');
const PORT = process.env.PORT || 3000;
const HOST = process.env.HOST || '0.0.0.0';
async function startServer() {
try {
// Connect to database
await connectDB();
console.log('✓ Database connected');
// Start server
const server = app.listen(PORT, HOST, () => {
console.log(`✓ Server running on http://${HOST}:${PORT}`);
});
// Graceful shutdown
process.on('SIGTERM', () => gracefulShutdown(server));
process.on('SIGINT', () => gracefulShutdown(server));
return server;
} catch (error) {
console.error('Failed to start server:', error);
process.exit(1);
}
}
function gracefulShutdown(server) {
console.log('Received shutdown signal. Closing connections...');
server.close(() => {
console.log('Server closed');
process.exit(0);
});
// Force close after 10s
setTimeout(() => {
console.error('Forced shutdown');
process.exit(1);
}, 10000);
}
startServer();
```
### 3. Routing Patterns
```javascript
// backend/src/routes/api/index.js
const express = require('express');
const router = express.Router();
const users = require('./users');
const posts = require('./posts');
const { validateRequest } = require('../../middleware/validation');
// Resource routes
router.use('/users', users);
router.use('/posts', posts);
// Validation middleware
router.use(validateRequest);
module.exports = router;
// backend/src/routes/api/users.js
const express = require('express');
const router = express.Router();
const userController = require('../../controllers/userController');
const { authenticate, authorize } = require('../../middleware/auth');
const { validateUser } = require('../../middleware/validation');
// Public routes
router.post('/register', validateUser, userController.register);
router.post('/login', userController.login);
// Protected routes
router.use(authenticate);
router.get('/me', userController.getProfile);
router.put('/me', validateUser, userController.updateProfile);
// Admin routes
router.get('/', authorize('admin'), userController.listUsers);
module.exports = router;
```
### 4. Controller Pattern
```javascript
// backend/src/controllers/userController.js
const userService = require('../services/userService');
const { AppError } = require('../middleware/error');
class UserController {
async register(req, res, next) {
try {
const { email, password, name } = req.body;
const user = await userService.create({ email, password, name });
res.status(201).json({
status: 'success',
data: { user: user.toSafeObject() }
});
} catch (error) {
next(error);
}
}
async getProfile(req, res, next) {
try {
const user = await userService.findById(req.user.id);
if (!user) {
throw new AppError('User not found', 404);
}
res.json({
status: 'success',
data: { user: user.toSafeObject() }
});
} catch (error) {
next(error);
}
}
}
module.exports = new UserController();
```
### 5. Service Layer Pattern
```javascript
// backend/src/services/userService.js
const User = require('../models/User');
const { hashPassword, comparePassword } = require('../utils/password');
const { generateToken } = require('../utils/jwt');
class UserService {
async create(userData) {
// Check if user exists
const existingUser = await User.findByEmail(userData.email);
if (existingUser) {
throw new AppError('Email already registered', 409);
}
// Hash password
const hashedPassword = await hashPassword(userData.password);
// Create user
const user = await User.create({
...userData,
password: hashedPassword
});
return user;
}
async authenticate(email, password) {
const user = await User.findByEmail(email);
if (!user) {
throw new AppError('Invalid credentials', 401);
}
const isValid = await comparePassword(password, user.password);
if (!isValid) {
throw new AppError('Invalid credentials', 401);
}
const token = generateToken({ id: user.id, role: user.role });
return { user, token };
}
}
module.exports = new UserService();
```
### 6. Async Handler Wrapper
```javascript
// backend/src/middleware/asyncHandler.js
const asyncHandler = (fn) => (req, res, next) => {
Promise.resolve(fn(req, res, next)).catch(next);
};
module.exports = asyncHandler;
// Usage
router.get('/users/:id', asyncHandler(userController.getById));
```
### 7. Request Validation
```javascript
// backend/src/middleware/validation.js
const { validationResult, body, param, query } = require('express-validator');
const validate = (validations) => {
return async (req, res, next) => {
await Promise.all(validations.map(validation => validation.run(req)));
const errors = validationResult(req);
if (errors.isEmpty()) {
return next();
}
res.status(400).json({
status: 'fail',
errors: errors.array().map(err => ({
field: err.path,
message: err.msg
}))
});
};
};
// Validation schemas
const userValidation = {
register: [
body('email')
.isEmail()
.normalizeEmail()
.withMessage('Valid email required'),
body('password')
.isLength({ min: 8 })
.withMessage('Password must be at least 8 characters'),
body(

View File

@@ -0,0 +1,377 @@
# NodeJS Middleware Patterns
Comprehensive middleware patterns for Express.js applications.
## Middleware Types
### 1. Application-level Middleware
```javascript
// Global middleware
app.use(express.json());
app.use(express.urlencoded({ extended: true }));
app.use(helmet());
app.use(cors());
// Conditional middleware
app.use((req, res, next) => {
if (req.path.startsWith('/api')) {
return apiLimiter(req, res, next);
}
next();
});
```
### 2. Router-level Middleware
```javascript
const router = require('express').Router();
// Router-specific middleware
router.use(authenticate);
router.use(validateRequest);
// Route-specific middleware
router.post('/', validateUser, controller.create);
router.put('/:id', validateUser, authenticate, controller.update);
```
### 3. Error-handling Middleware
```javascript
// Must have 4 parameters
app.use((err, req, res, next) => {
console.error(err.stack);
res.status(err.statusCode || 500).json({
status: 'error',
message: err.message
});
});
```
### 4. Built-in Middleware
```javascript
app.use(express.static('public'));
app.use(express.json());
app.use(express.urlencoded({ extended: true }));
```
## Common Patterns
### Authentication Middleware
```javascript
function authenticate(req, res, next) {
const authHeader = req.headers.authorization;
if (!authHeader?.startsWith('Bearer ')) {
return res.status(401).json({ message: 'Unauthorized' });
}
const token = authHeader.split(' ')[1];
try {
const decoded = verifyToken(token);
req.user = decoded;
next();
} catch (error) {
return res.status(401).json({ message: 'Invalid token' });
}
}
```
### Authorization Middleware
```javascript
function authorize(...roles) {
return (req, res, next) => {
if (!roles.includes(req.user.role)) {
return res.status(403).json({ message: 'Forbidden' });
}
next();
};
}
// Usage
router.delete('/:id', authenticate, authorize('admin'), controller.delete);
```
### Rate Limiting Middleware
```javascript
const rateLimit = require('express-rate-limit');
const limiter = rateLimit({
windowMs: 15 * 60 * 1000,
max: 100,
standardHeaders: true,
legacyHeaders: false,
message: { message: 'Too many requests' }
});
app.use('/api', limiter);
```
### Validation Middleware
```javascript
const { validationResult, body } = require('express-validator');
function validate(validations) {
return async (req, res, next) => {
await Promise.all(validations.map(v => v.run(req)));
const errors = validationResult(req);
if (errors.isEmpty()) return next();
res.status(400).json({
status: 'fail',
errors: errors.array().map(err => ({
field: err.path,
message: err.msg
}))
});
};
}
// Usage
router.post('/users',
validate([
body('email').isEmail(),
body('password').isLength({ min: 8 })
]),
controller.create
);
```
### Request Logging Middleware
```javascript
const pino = require('pino');
const logger = pino();
function requestLogger(req, res, next) {
const start = Date.now();
res.on('finish', () => {
const duration = Date.now() - start;
logger.info({
method: req.method,
path: req.path,
status: res.statusCode,
duration,
user: req.user?.id
});
});
next();
}
```
### Context Middleware
```javascript
// Request-scoped context
const asyncLocalStorage = new (require('async_hooks').AsyncLocalStorage)();
function contextMiddleware(req, res, next) {
const context = {
requestId: require('uuid').v4(),
userId: req.user?.id,
startTime: Date.now()
};
asyncLocalStorage.run(context, next);
}
// Access context anywhere in the request lifecycle
function getContext() {
return asyncLocalStorage.getStore();
}
```
### Error Handler Middleware
```javascript
function errorHandler(err, req, res, next) {
const statusCode = err.statusCode || 500;
const status = err.status || 'error';
if (process.env.NODE_ENV === 'development') {
return res.status(statusCode).json({
status,
message: err.message,
stack: err.stack
});
}
if (err.isOperational) {
return res.status(statusCode).json({
status,
message: err.message
});
}
res.status(500).json({
status: 'error',
message: 'Something went wrong'
});
}
```
## Third-party Middleware
### Security (Helmet)
```javascript
const helmet = require('helmet');
app.use(helmet());
app.use(helmet.contentSecurityPolicy({
directives: {
defaultSrc: ["'self'"],
scriptSrc: ["'self'"]
}
}));
```
### Compression
```javascript
const compression = require('compression');
app.use(compression({
threshold: 1024, // Only compress >1KB
filter: (req, res) => {
if (req.headers['x-no-compression']) return false;
return compression.filter(req, res);
}
}));
```
### CORS
```javascript
const cors = require('cors');
app.use(cors({
origin: process.env.CORS_ORIGIN?.split(',') || '*',
methods: ['GET', 'POST', 'PUT', 'DELETE'],
allowedHeaders: ['Content-Type', 'Authorization'],
credentials: true
}));
```
### Morgan (Logging)
```javascript
const morgan = require('morgan');
app.use(morgan('dev')); // Development
app.use(morgan('combined')); // Production
```
## Custom Middleware Factory
```javascript
// Custom middleware factory
function createMiddleware(options) {
return (req, res, next) => {
// Pre-processing
if (options.preCondition && !options.preCondition(req)) {
return res.status(400).json({ message: 'Precondition failed' });
}
// Modify request
req.custom = options.transform?.(req) || req.custom;
// Post-processing (on response)
res.on('finish', () => {
options.postProcess?.(req, res);
});
next();
};
}
// Usage
app.use(createMiddleware({
preCondition: (req) => req.headers['x-api-key'],
transform: (req) => ({ ip: req.ip }),
postProcess: (req, res) => console.log(`Request completed: ${req.path}`)
}));
```
## Async Middleware Wrapper
```javascript
function asyncHandler(fn) {
return (req, res, next) =>
Promise.resolve(fn(req, res, next)).catch(next);
}
// Wrap middleware
app.get('/users/:id', asyncHandler(async (req, res) => {
const user = await User.findById(req.params.id);
res.json({ user });
}));
```
## Middleware Composition
```javascript
// Compose multiple middleware
function compose(...middlewares) {
return (req, res, next) => {
let index = -1;
function dispatch(i) {
if (i <= index) return next(new Error('next() called multiple times'));
index = i;
const middleware = middlewares[i];
if (!middleware) return next();
return middleware(req, res, (err) => {
if (err) return next(err);
return dispatch(i + 1);
});
}
return dispatch(0);
};
}
// Usage
app.use(compose(
authenticate,
authorize('user'),
validateRequest
));
```
## Best Practices
1. **Order matters** - Put error handlers last
2. **Use router middleware** - For route groups
3. **Use async handlers** - Wrap async functions
4. **Keep it simple** - One responsibility per middleware
5. **Use third-party** - Don't reinvent the wheel
6. **Document dependencies** - Comment required fields
## Middleware Order
```javascript
// Correct order
app.use(helmet()); // Security headers
app.use(cors()); // CORS
app.use(compression()); // Compression
app.use(express.json()); // Body parsing
app.use(express.urlencoded()); // URL encoding
app.use(morgan('dev')); // Logging
app.use('/api', apiLimiter); // Rate limiting
app.use('/api', routes); // Routes
app.use(notFoundHandler); // 404 handler
app.use(errorHandler); // Error handler
```

View File

@@ -0,0 +1,378 @@
# NodeJS npm Management
Best practices for package management in Node.js projects.
## package.json Best Practices
```json
{
"name": "my-project",
"version": "1.0.0",
"description": "Project description",
"main": "src/index.js",
"scripts": {
"start": "node src/server.js",
"dev": "nodemon src/server.js",
"build": "tsc",
"test": "jest",
"test:coverage": "jest --coverage",
"lint": "eslint src/",
"lint:fix": "eslint src/ --fix",
"format": "prettier --write \"src/**/*.js\"",
"audit": "npm audit",
"audit:fix": "npm audit fix",
"db:migrate": "knex migrate:latest",
"db:rollback": "knex migrate:rollback",
"db:seed": "knex seed:run"
},
"dependencies": {
"express": "^4.18.2",
"helmet": "^7.1.0",
"pg": "^8.11.3"
},
"devDependencies": {
"eslint": "^8.56.0",
"jest": "^29.7.0",
"nodemon": "^3.0.2",
"prettier": "^3.2.0"
},
"engines": {
"node": ">=20.0.0",
"npm": ">=10.0.0"
},
"keywords": ["nodejs", "express", "api"],
"author": "Author Name",
"license": "MIT",
"repository": {
"type": "git",
"url": "git+https://github.com/user/repo.git"
}
}
```
## Semantic Versioning
```
MAJOR.MINOR.PATCH
MAJOR: Breaking changes
MINOR: New features, backward compatible
PATCH: Bug fixes, backward compatible
Prefixes:
^1.2.3 -> >=1.2.3 <2.0.0 (allows MINOR and PATCH updates)
~1.2.3 -> >=1.2.3 <1.3.0 (allows PATCH updates only)
1.2.3 -> exactly 1.2.3 (locked)
```
## Scripts
### Development Scripts
```json
{
"scripts": {
"start": "node src/server.js",
"dev": "NODE_ENV=development nodemon src/server.js",
"dev:debug": "NODE_ENV=development nodemon --inspect src/server.js"
}
}
```
### Testing Scripts
```json
{
"scripts": {
"test": "jest",
"test:watch": "jest --watch",
"test:coverage": "jest --coverage",
"test:ci": "jest --ci --coverage --reporters=default --reporters=jest-junit"
}
}
```
### Build Scripts
```json
{
"scripts": {
"build": "tsc",
"build:prod": "tsc && npm run db:migrate",
"clean": "rm -rf dist/ node_modules/",
"reinstall": "rm -rf node_modules/ && npm install"
}
}
```
### Linting Scripts
```json
{
"scripts": {
"lint": "eslint src/",
"lint:fix": "eslint src/ --fix",
"format": "prettier --write \"src/**/*.js\"",
"format:check": "prettier --check \"src/**/*.js\""
}
}
```
## Lockfile Management
```bash
# Generate lockfile
npm install
# Update lockfile
npm install package-name
# Clean install (uses lockfile)
npm ci
# Update all packages
npm update
# Update specific package
npm update package-name
```
## Security Audits
```bash
# Check for vulnerabilities
npm audit
# Fix vulnerabilities
npm audit fix
# Force fix breaking changes
npm audit fix --force
# Show details
npm audit --json
```
## .npmrc Configuration
```
# .npmrc
registry=https://registry.npmjs.org/
# Auth token for private packages
//npm.pkg.github.com/:_authToken=${NPM_TOKEN}
@mycompany:registry=https://npm.pkg.github.com
# Cache
cache=.npm
# Strict SSL
strict-ssl=true
# Progress
progress=false
```
## ESLint Configuration
```javascript
// .eslintrc.js
module.exports = {
env: {
node: true,
es2021: true,
jest: true
},
extends: [
'eslint:recommended'
],
parserOptions: {
ecmaVersion: 'latest',
sourceType: 'module'
},
rules: {
'no-unused-vars': 'error',
'no-console': 'warn',
'no-constant-condition': 'error',
'prefer-const': 'error',
'no-var': 'error',
'eqeqeq': ['error', 'always'],
'curly': ['error', 'multi-line']
}
};
```
## Prettier Configuration
```javascript
// .prettierrc.js
module.exports = {
semi: true,
singleQuote: true,
tabWidth: 2,
trailingComma: 'none',
printWidth: 100,
bracketSpacing: true,
arrowParens: 'avoid'
};
```
## TypeScript Configuration
```json
// tsconfig.json
{
"compilerOptions": {
"target": "ES2022",
"module": "commonjs",
"lib": ["ES2022"],
"outDir": "./dist",
"rootDir": "./src",
"strict": true,
"esModuleInterop": true,
"skipLibCheck": true,
"forceConsistentCasingInFileNames": true,
"resolveJsonModule": true,
"moduleResolution": "node",
"sourceMap": true
},
"include": ["src/**/*"],
"exclude": ["node_modules", "dist", "**/*.test.ts"]
}
```
## Dependencies Best Practices
### Check for outdated
```bash
npm outdated
# Output:
# Package Current Wanted Latest Location
# express 4.17.1 4.18.2 4.18.2 my-project
```
### Check for unused
```bash
npx depcheck
# Output:
# Unused dependencies
# * moment
# Unused devDependencies
# * sinon
```
### Bundle size
```bash
npx bundlesize
# Check bundle size limits
```
## Monorepo with npm workspaces
```json
// package.json (root)
{
"workspaces": [
"packages/*"
]
}
```
```
project/
├── package.json
├── packages/
│ ├── api/
│ │ └── package.json
│ └── web/
│ └── package.json
└── node_modules/
```
### Workspace commands
```bash
# Install all workspaces
npm install
# Install in specific workspace
npm install package-name -w packages/api
# Run script in workspace
npm run test -w packages/api
# Add dependency to workspace
npm install express -w packages/api
```
## Publishing
```bash
# Login
npm login
# Bump version
npm version minor # or patch, major
# Dry run
npm pack --dry-run
# Publish
npm publish
# Publish as scoped package
npm publish --access public
```
## .npmignore
```
# .npmignore
tests/
coverage/
.github/
.eslintrc.js
.prettierrc
tsconfig.json
*.test.js
*.spec.js
.DS_Store
.env*
!.env.example
```
## Quick Reference
```bash
# Install package
npm install package-name # production
npm install -D package-name # dev
npm install -g package-name # global
# Remove package
npm uninstall package-name
# Update
npm update # all
npm update package-name # specific
# Run script
npm run script-name
# Run binary
npx package-name
# List dependencies
npm list # all
npm list --depth=0 # top-level only
npm list -g --depth=0 # global
# View package info
npm view package-name
npm view package-name versions```

View File

@@ -0,0 +1,335 @@
# NodeJS Security (OWASP)
OWASP Top 10 security practices for Node.js applications.
## OWASP Top 10 for Node.js
### 1. Injection (A03:2021)
**SQL Injection Prevention:**
```javascript
// ❌ Vulnerable
const query = `SELECT * FROM users WHERE id = ${id}`;
// ✅ Parameterized queries
const query = 'SELECT * FROM users WHERE id = ?';
db.query(query, [id]);
// ✅ Use ORM with escaping
const user = await User.query().where('id', id).first();
```
**NoSQL Injection:**
```javascript
// ❌ Vulnerable
const user = await User.find({ email: req.body.email });
// ✅ Validate and sanitize
const { email } = req.body;
if (typeof email !== 'string') {
throw new AppError('Invalid input', 400);
}
const user = await User.find({ email: sanitize(email) });
```
### 2. Broken Authentication (A07:2021)
```javascript
// ✅ Password hashing
const bcrypt = require('bcrypt');
const hashedPassword = await bcrypt.hash(password, 12);
// ✅ Password comparison
await bcrypt.compare(password, hashedPassword);
// ✅ Session security
const session = require('express-session');
app.use(session({
secret: process.env.SESSION_SECRET,
cookie: {
httpOnly: true,
secure: true, // HTTPS only
sameSite: 'strict',
maxAge: 3600000 // 1 hour
}
}));
```
### 3. Sensitive Data Exposure (A02:2021)
```javascript
// ✅ Environment variables
require('dotenv').config();
const dbPassword = process.env.DB_PASSWORD;
// ✅ Encrypt sensitive data
const crypto = require('crypto');
const algorithm = 'aes-256-gcm';
const key = crypto.scryptSync(process.env.SECRET, 'salt', 32);
function encrypt(text) {
const iv = crypto.randomBytes(16);
const cipher = crypto.createCipheriv(algorithm, key, iv);
const encrypted = Buffer.concat([cipher.update(text), cipher.final()]);
return { iv: iv.toString('hex'), data: encrypted.toString('hex') };
}
// ❌ Never log sensitive data
console.log('User data:', user); // BAD
// ✅ Sanitize logs
console.log('User logged in:', { id: user.id, timestamp: Date.now() });
```
### 4. XML External Entities (A05:2021)
```javascript
// ✅ Disable XML external entities
const xml2js = require('xml2js');
const parser = new xml2js.Parser({
explicitArray: false,
doctype: false, // Disable DOCTYPE
xmlns: false // Disable xmlns
});
```
### 5. Broken Access Control (A01:2021)
```javascript
// ✅ Role-based access control
function authorize(...roles) {
return (req, res, next) => {
if (!roles.includes(req.user.role)) {
return res.status(403).json({ message: 'Forbidden' });
}
next();
};
}
// Usage
router.delete('/users/:id', authenticate, authorize('admin'), deleteUser);
// ✅ Resource ownership check
async function canAccessResource(req, res, next) {
const resource = await Resource.findById(req.params.id);
if (resource.userId !== req.user.id && req.user.role !== 'admin') {
return res.status(403).json({ message: 'Forbidden' });
}
next();
}
```
### 6. Security Misconfiguration (A05:2021)
```javascript
// ✅ Use helmet for security headers
const helmet = require('helmet');
app.use(helmet());
// ✅ Content Security Policy
app.use(helmet.contentSecurityPolicy({
directives: {
defaultSrc: ["'self'"],
styleSrc: ["'self'", 'fonts.googleapis.com'],
fontSrc: ["'self'", 'fonts.gstatic.com'],
imgSrc: ["'self'", 'data:'],
scriptSrc: ["'self'"],
connectSrc: ["'self'", 'api.example.com'],
frameAncestors: ["'none'"]
}
}));
// ✅ Disable headers
app.disable('x-powered-by');
// ✅ CORS configuration
app.use(cors({
origin: process.env.CORS_ORIGIN,
methods: ['GET', 'POST', 'PUT', 'DELETE'],
allowedHeaders: ['Content-Type', 'Authorization'],
credentials: true,
maxAge: 86400
}));
```
### 7. Cross-Site Scripting (XSS) (A03:2021)
```javascript
// ✅ Sanitize user input
const sanitizeHtml = require('sanitize-html');
const clean = sanitizeHtml(userInput, {
allowedTags: ['b', 'i', 'em', 'strong'],
allowedAttributes: {}
});
// ✅ Escape output
const escapeHtml = (str) =>
str.replace(/&/g, '&amp;')
.replace(/</g, '&lt;')
.replace(/>/g, '&gt;')
.replace(/"/g, '&quot;')
.replace(/'/g, '&#039;');
// ✅ Set XSS protection header
app.use(helmet.xssFilter());
```
### 8. Insecure Deserialization (A08:2021)
```javascript
// ✅ Use JSON.parse with validation
function safeParse(jsonString, validator) {
try {
const data = JSON.parse(jsonString);
if (validator && !validator(data)) {
throw new Error('Invalid data structure');
}
return data;
} catch (error) {
throw new AppError('Invalid JSON', 400);
}
}
// ❌ Never use eval()
eval(userData); // NEVER DO THIS
// ✅ Safe alternatives
JSON.parse(userData);
Function('return ' + userData)(); // Also UNSAFE, don't use
```
### 9. Using Components with Known Vulnerabilities (A06:2021)
```javascript
// ✅ Regular security audits
// package.json
{
"scripts": {
"audit": "npm audit",
"audit:fix": "npm audit fix"
}
}
// Run: npm run audit
// Check: snyk, npm audit, OWASP Dependency-Check
// ✅ Keep dependencies updated
npm outdated
npm update
```
### 10. Insufficient Logging & Monitoring (A09:2021)
```javascript
// ✅ Structured logging
const pino = require('pino');
const logger = pino({ level: 'info' });
// Log security events
app.post('/login', async (req, res) => {
const { email } = req.body;
const user = await authenticate(req.body);
logger.info({
event: 'user_login',
userId: user.id,
ip: req.ip,
userAgent: req.headers['user-agent'],
timestamp: new Date().toISOString()
});
res.json({ token: user.token });
});
// ✅ Alert on suspicious activity
function detectBruteForce(email, failures) {
if (failures > 5) {
logger.warn({
event: 'brute_force_attempt',
email,
ip: req.ip,
timestamp: new Date().toISOString()
});
}
}
```
## Additional Security Measures
### Rate Limiting
```javascript
const rateLimit = require('express-rate-limit');
// General limiter
const apiLimiter = rateLimit({
windowMs: 15 * 60 * 1000, // 15 min
max: 100
});
// Strict limiter for auth
const authLimiter = rateLimit({
windowMs: 60 * 60 * 1000, // 1 hour
max: 5
});
app.use('/api', apiLimiter);
app.post('/login', authLimiter, loginHandler);
```
### Input Validation
```javascript
const { body, validationResult } = require('express-validator');
app.post('/users',
body('email').isEmail().normalizeEmail(),
body('password').isLength({ min: 8 }),
body('name').trim().escape(),
(req, res) => {
const errors = validationResult(req);
if (!errors.isEmpty()) {
return res.status(400).json({ errors: errors.array() });
}
// Process...
}
);
```
### Secret Management
```javascript
// ✅ Use environment variables
const jwt = require('jsonwebtoken');
const token = jwt.sign(payload, process.env.JWT_SECRET, { expiresIn: '1h' });
// ✅ Never hardcode secrets
const API_KEY = 'sk-abc123'; // NEVER DO THIS
// ✅ Use .env files (not committed)
JWT_SECRET=your-secret-here
API_KEY=your-api-key
// ✅ Use vault services in production
// HashiCorp Vault, AWS Secrets Manager, etc.
```
## Security Checklist
- [ ] Use helmet middleware
- [ ] Enable CORS properly
- [ ] Rate limit all public endpoints
- [ ] Validate all input
- [ ] Use parameterized queries
- [ ] Hash passwords with bcrypt
- [ ] Use HTTPS
- [ ] Implement JWT correctly
- [ ] Log security events
- [ ] Run npm audit regularly
- [ ] Keep dependencies updated
- [ ] Use environment variables for secrets
- [ ] Implement proper error handling
- [ ] Add Content Security Policy
- [ ] Disable unnecessary headers

View File

@@ -0,0 +1,412 @@
# NodeJS Testing with Jest
Comprehensive testing patterns for Node.js applications using Jest.
## Setup
```bash
npm install --save-dev jest @types/jest ts-jest
```
```json
// package.json
{
"scripts": {
"test": "jest",
"test:watch": "jest --watch",
"test:coverage": "jest --coverage"
}
}
```
```javascript
// jest.config.js
module.exports = {
testEnvironment: 'node',
roots: ['<rootDir>/tests'],
testMatch: ['**/*.test.js'],
collectCoverageFrom: [
'src/**/*.js',
'!src/**/*.d.ts'
],
coverageThreshold: {
global: {
branches: 80,
functions: 80,
lines: 80,
statements: 80
}
}
};
```
## Test Structure
```
tests/
├── unit/
│ ├── services/
│ │ └── userService.test.js
│ ├── controllers/
│ │ └── userController.test.js
│ └── utils/
│ └── helpers.test.js
├── integration/
│ ├── api/
│ │ └── users.test.js
│ └── db/
│ └── connection.test.js
└── mocks/
└── mockData.js
```
## Unit Tests
### Service Tests
```javascript
// tests/unit/services/userService.test.js
const userService = require('../../../src/services/userService');
const User = require('../../../src/models/User');
const { hashPassword } = require('../../../src/utils/password');
// Mock dependencies
jest.mock('../../../src/models/User');
jest.mock('../../../src/utils/password');
describe('UserService', () => {
beforeEach(() => {
jest.clearAllMocks();
});
describe('create', () => {
it('should create a new user with hashed password', async () => {
// Arrange
const userData = {
email: 'test@example.com',
password: 'password123',
name: 'Test User'
};
const hashedPassword = 'hashed_password';
const createdUser = { id: 1, ...userData, password: hashedPassword };
User.findByEmail.mockResolvedValue(null);
hashPassword.mockResolvedValue(hashedPassword);
User.create.mockResolvedValue(createdUser);
// Act
const result = await userService.create(userData);
// Assert
expect(User.findByEmail).toHaveBeenCalledWith(userData.email);
expect(hashPassword).toHaveBeenCalledWith(userData.password);
expect(User.create).toHaveBeenCalledWith({
...userData,
password: hashedPassword
});
expect(result).toEqual(createdUser);
});
it('should throw error if email already exists', async () => {
// Arrange
const userData = { email: 'existing@example.com', password: 'pass' };
User.findByEmail.mockResolvedValue({ id: 1 });
// Act & Assert
await expect(userService.create(userData))
.rejects.toThrow('Email already registered');
});
});
});
```
### Controller Tests
```javascript
// tests/unit/controllers/userController.test.js
const userController = require('../../../src/controllers/userController');
const userService = require('../../../src/services/userService');
jest.mock('../../../src/services/userService');
describe('UserController', () => {
let req, res, next;
beforeEach(() => {
req = { body: {}, params: {}, user: {} };
res = { status: jest.fn().mockReturnThis(), json: jest.fn() };
next = jest.fn();
jest.clearAllMocks();
});
describe('register', () => {
it('should register user and return 201', async () => {
// Arrange
req.body = { email: 'test@example.com', password: 'password123' };
const user = { id: 1, email: 'test@example.com' };
userService.create.mockResolvedValue(user);
// Act
await userController.register(req, res, next);
// Assert
expect(res.status).toHaveBeenCalledWith(201);
expect(res.json).toHaveBeenCalledWith({
status: 'success',
data: { user }
});
expect(next).not.toHaveBeenCalled();
});
it('should call next with error on failure', async () => {
// Arrange
const error = new Error('Database error');
userService.create.mockRejectedValue(error);
// Act
await userController.register(req, res, next);
// Assert
expect(next).toHaveBeenCalledWith(error);
});
});
});
```
## Integration Tests
### API Tests
```javascript
// tests/integration/api/users.test.js
const request = require('supertest');
const app = require('../../../src/app');
const User = require('../../../src/models/User');
// Setup/Teardown
beforeAll(async () => {
await connectTestDatabase();
});
afterAll(async () => {
await disconnectTestDatabase();
});
beforeEach(async () => {
await User.deleteMany({});
});
describe('Users API', () => {
describe('POST /api/users/register', () => {
it('should register a new user', async () => {
const response = await request(app)
.post('/api/users/register')
.send({
email: 'test@example.com',
password: 'password123',
name: 'Test User'
})
.expect(201);
expect(response.body.status).toBe('success');
expect(response.body.data.user).toHaveProperty('id');
expect(response.body.data.user.email).toBe('test@example.com');
});
it('should return 400 for invalid email', async () => {
const response = await request(app)
.post('/api/users/register')
.send({
email: 'invalid-email',
password: 'password123'
})
.expect(400);
expect(response.body.status).toBe('fail');
});
});
describe('GET /api/users/me', () => {
it('should return user profile when authenticated', async () => {
// Create user and get token
const user = await User.create({
email: 'test@example.com',
password: await hashPassword('password123')
});
const token = generateToken({ id: user.id });
const response = await request(app)
.get('/api/users/me')
.set('Authorization', `Bearer ${token}`)
.expect(200);
expect(response.body.data.user.email).toBe('test@example.com');
});
it('should return 401 when not authenticated', async () => {
await request(app)
.get('/api/users/me')
.expect(401);
});
});
});
```
## Mocking
### Mock Modules
```javascript
// Mock external module
jest.mock('axios');
// Mock internal module
jest.mock('../../../src/services/emailService', () => ({
sendEmail: jest.fn()
}));
// Mock environment variable
process.env.JWT_SECRET = 'test-secret';
```
### Mock Functions
```javascript
// Create mock function
const mockFn = jest.fn();
// Set return value
mockFn.mockReturnValue('value');
mockFn.mockReturnValueOnce('value once');
// Set implementation
mockFn.mockImplementation((x) => x * 2);
// Set resolved value (async)
mockFn.mockResolvedValue({ data: 'value' });
mockFn.mockRejectedValue(new Error('failed'));
// Check calls
expect(mockFn).toHaveBeenCalled();
expect(mockFn).toHaveBeenCalledTimes(2);
expect(mockFn).toHaveBeenCalledWith('arg1', 'arg2');
expect(mockFn).toHaveReturnedWith('value');
```
## Test Coverage
```javascript
// Run coverage
npm run test:coverage
// Coverage report
// tests/unit/services/userService.test.js
// File | % Stmts | % Branch | % Funcs | % Lines |
//--------------|---------|----------|---------|---------|
// userService | 95.45 | 87.50 | 100.00 | 95.45 |
```
## Best Practices
### Don't Do This
```javascript
// ❌ Test implementation details
it('should set isLoading to true', () => {
component.setState({ isLoading: true });
expect(component.state().isLoading).toBe(true);
});
// ❌ Test everything in one test
it('should work', async () => {
await request(app).post('/users').send(data);
await request(app).get('/users/1');
// ...many more assertions
});
// ❌ Use real database in unit tests
const db = require('mongoose').connect('mongodb://localhost/test');
```
### Do This
```javascript
// ✅ Test behavior
it('should show loading indicator', () => {
render(<Component />);
expect(screen.getByRole('progressbar')).toBeInTheDocument();
});
// ✅ Test one thing per test
it('should create user', async () => {
await request(app).post('/users').send(data).expect(201);
});
it('should return created user', async () => {
const res = await request(app).post('/users').send(data);
expect(res.body.data.user.email).toBe(data.email);
});
// ✅ Use mock database
jest.mock('../../../src/db/connection');
```
## Testing Async Code
```javascript
// Promises
it('should resolve with data', async () => {
const result = await service.getData();
expect(result).toEqual(expectedData);
});
// Callbacks
it('should call callback', (done) => {
service.asyncOperation((result) => {
expect(result).toBe('expected');
done();
});
});
// Timers
jest.useFakeTimers();
it('should timeout after 5s', () => {
const callback = jest.fn();
setTimeout(callback, 5000);
jest.advanceTimersByTime(5000);
expect(callback).toHaveBeenCalled();
});
```
## Test Fixtures
```javascript
// tests/fixtures/users.js
module.exports = {
validUser: {
email: 'test@example.com',
password: 'Password123!',
name: 'Test User'
},
invalidUser: {
email: 'invalid',
password: 'short'
},
adminUser: {
email: 'admin@example.com',
password: 'Admin123!',
name: 'Admin User',
role: 'admin'
}
};
// Usage
const { validUser, invalidUser } = require('../fixtures/users');
```
## See Also
- `nodejs-express-patterns` - Express patterns
- `nodejs-auth-jwt` - Authentication testing
- `nodejs-error-handling` - Error scenarios