Document design decisions under `docs/`

[yamanidev]

Is your feature request related to a problem? Please describe. There is currently no centralized place to track the design decisions made throughout the project. This makes it difficult for new contributors or team members to understand the reasoning behind certain choices.

Describe the solution you'd like Document the design decisions made during the development of the project in a docs/ folder. This will serve as a reference to understand why specific design choices were made and help maintain consistency across the project.

Additional context An example of how the documentation could look like is Epic Stack's.

[omdxp]

What about the discussions page?

[yamanidev]

That could be helpful as an addition, but having a single source of truth for the existing design decisions made, I think it would be best if they're documented separately under docs/ for example.

What do you think?

[omdxp]

I like this idea, having a single source of truth is always good and it would help new comers what the project went through and how it reached this point, wdyt @ZibanPirate ?

[ZibanPirate]

i like the idea, i'm fine with this as long as we have a single file that documents everything

i'm against having multiple files, one per decision

[ZibanPirate]

now that I think about it, it may make sense to document these decisions in the ./CONTRIBUTING.md file, since it is used by GitHub to guide contributors

[yamanidev]

I don't think so. CONTRIBUTING.md is usually guidelines on how to contribute, about certain conventions and whatnot, not the technical decisions that were made for the given project.

wdyt?

[ZibanPirate]

My understanding is: the idea of "documenting decisions" did not exist out of vacuum, but instead we need it to "help contributing to the repo", and if that is the reason, then it makes sense to be in the CONTRIBUTING.md file (which GitHub uses as part of their contribution UX)

Or are there other reasons (that interest us) for documenting decisions?

[yamanidev]

It's definitely the case, it's only relevant for someone who's interested in the project and especially contributing to it, I just have trouble imagining myself checking the CONTRIBUTIONS.md file for this kind of information, it doesn't sound intuitive.

For a start, I don't think it would matter where we put this information, we could always explicitly point it out in the README.md that it's in the CONTRIBUTIONS.md file, and then if this becomes troublesome to manage later, we could discuss it again.

[ZibanPirate]

first decision to be recorded: https://github.com/dzcode-io/dzcode.io/blob/3341188453e1fc8ad7baa94de2a7cae1be5b7cd3/.github/CONTRIBUTING.md#design-decisions

is this how it's supposed to be used?

[omdxp]

I've stumbled upon this project https://git-cliff.org which generates CHANGELOG.md simply based on commits with:

git cliff > CHANGELOG.md

I thought it's helpful for this issue.

Tested it locally and this was the output

# Changelog

All notable changes to this project will be documented in this file.

## [unreleased]

### 🐛 Bug Fixes

- Add pull request trigger to pull_request_target for CI workflow

## [6.9.2] - 2025-02-06

### 🚀 Features

- Add cleanup for old Docker images and containers in deployment script

### 🐛 Bug Fixes

- Update MeiliSearch image version and add service dependencies in Docker Compose
- Reorder cleanup logs for old Docker images and containers in deployment script

### 🚜 Refactor

- Remove old container management logic from deployment script

## [6.9.1] - 2025-01-25

### 🚜 Refactor

- Update language imports and types in sitemap functions

## [6.8.1] - 2024-12-31

### 🚀 Features

- Add script to prune unused keys from dictionary and update package.json

### 🐛 Bug Fixes

- Switch languages at root

### 🚜 Refactor

- Standardize API endpoint names to lowercase
- Standardize controller and dictionary endpoint names to lowercase
- Remove unused dictionary entries and clean up code
- Remove TODO comments for adding canonical URLs on contribution and project pages

## [6.8.0] - 2024-12-30

### 🚀 Features

- Add JSON schema for project model with required properties and tags
- Add tags and JSON schema reference to project info files
- Save TagEntity for each project
- Tag to dzcode.io website as solve-algerian-problem
- Add endpoint to retrieve contribution title by ID
- Implement contribution page metadata
- Add XML escaping for contribution URLs in sitemap
- Added hey cli project

### 🐛 Bug Fixes

- Improve wording for project tags in dictionary
- Add TODO comment to make project listing configurable
- Prevent rendering empty project groups in the project listing

### 💼 Other

- Id
- Id page

### 🚜 Refactor

- Remove TODO comment regarding title rendering in sitemap contributions
- Add TODO for SQL injection guard in findTitle method

## [6.7.0] - 2024-12-24

### 🚀 Features

- Add deleteAllDocuments method to SearchService for bulk document deletion
- Enhance DigestCron to integrate search service for indexing and deletion of project, contribution, and contributor items
- Update DigestCron to modify project ID format for MeiliSearch compatibility
- Refactor DigestCron to use upsert method for search indexing and remove bulk deletion
- Enhance DigestCron and SearchService to support bulk upsert and deletion of search items with runId
- Refactor DigestCron to streamline upsert operations and remove unused search item arrays
- Update GetSearchResponse to use MultiSearchResponse and simplify SearchItem type
- Update search endpoint to use POST method with request body and improve search functionality
- Enhance search functionality by specifying attributes to retrieve for each index
- Change search endpoint to use GET method with query parameters
- Add search input to top bar with localization support
- Implement search modal and enhance top bar with search functionality
- Enhance search functionality and update contribution types for better data retrieval
- Add localized labels for search contributions, contributors, and projects
- Update top bar to include a search button and improve modal display logic
- Enhance search component with loading state and localized messages for better user experience
- Add query parameters type to Search endpoint for enhanced readability
- Implement useSearchModal hook for improved modal handling in Search and TopBar components

### 🐛 Bug Fixes

- Improve readability and correct typos in README.md
- Enhance clarity and correct typos in API README.md
- Improve clarity and correct typos in web README.md
- Update Arabic translations for improved accuracy in dictionary.ts
- Improve wording and clarity in pull request template
- Improve wording for clarity in question issue template
- Improve wording and consistency in release configuration files

### 🚜 Refactor

- Rename setupIndexes to setupSearch for clarity and streamline search service initialization
- Update entity types in DigestCron and SearchService for consistency and clarity
- Remove redundant type assertion in contributionEntity creation
- Streamline index setup by introducing upsertIndex method for improved index management
- Remove default value for runId in contributors and projects tables for consistency
- Remove logging of search results count in SearchService
- Update contribution, contributor, and project types for improved clarity and structure
- Update search component to use entity models for improved type safety and clarity
- Update import paths for project, contributor, and contribution cards for consistency
- Move keyboard event handling to Search component for better encapsulation
- Update logo import and adjust button sizes in TopBar for improved layout consistency
- Rename contribute-card to contribution-card and update import paths for consistency
- Update search component to use typed responses for projects, contributors, and contributions
- Update TopBar search input to be read-only and enhance click behavior
- Simplify results handling in Search component and initialize state with an empty array
- Move onKeyDown function inside useEffect for better encapsulation
- Update ContributorCard and TopBar components for consistency and bug fix

### ⚙️ Miscellaneous Tasks

- Introduce setupIndexes search method that ensure indexes and add update filterable attributes
- Update change template in release drafter to include author information
- Add full changelog link to release drafter configuration
- Address some comments

## [6.6.0] - 2024-12-17

### 🚀 Features

- Added black contributor page
- Fetch contor details from api
- Shows stats on contributor details page
- Projects a contributor helped in
- Contributions a contributor needs helped with

### 🐛 Bug Fixes

- Removed rspack doctor for security

## [6.4.0] - 2024-09-30

### ⚙️ Miscellaneous Tasks

- Fun project name

## [6.2.0] - 2024-09-17

### 💼 Other

- Upload all artis merged

## [stg-v6.1.0] - 2024-09-09

### 🚀 Features

- Setup digest cron
- Setup sqlite database with an ts friendly orm
- Type-checked table against model

### 🚜 Refactor

- Remove unused code and dependencies in team page

### ⚙️ Miscellaneous Tasks

- Upped sentry on `./api`
- Upgraded prettier
- Don't export GithubAccount class
- Update NODE_ENV validation in ENVDto

## [stg-v6.0.0] - 2024-09-03

### 🚀 Features

- Use jest config from tooling package

### ⚙️ Miscellaneous Tasks

- Update CodeQL analysis workflow to include TypeScript support
- Remove outdated README.md file
- Defined typeof STAGE globally
- Moved footerSections instantiation to module level
- Moved topBarLinks instantiation to module level
- Better error handling

## [stg-v5.3.6-pr591.1] - 2024-09-02

### 🚀 Features

- Reported caught errors to sentry
- 404 page

### 🐛 Bug Fixes

- Navbar selection bug

### ⚙️ Miscellaneous Tasks

- Removed unused files
- Put back deploy script for web
- Removed unused files
- Max-width xl
- Added SEO and GA
- Temp deploy
- Generate HTML files during CI build for @dzcode.io/web
- Hide googleAnalytics when on dev
- Added sentry back to web
- Tooling fixes
- Upped firebase-tools
- Migrate web

## [stg-v5.3.6-pr591.0] - 2024-08-31

### 🚀 Features

- Add functionFactory for localized text rendering
- Add FAQ page data and render it dynamically
- Setup unit tests with coverage
- Deploy on push to rspack
- Update dev scripts
- Added team page
- Added contribute page
- Added milestones to landing page

### 🐛 Bug Fixes

- Make ts-node work on Windows
- One simple e2e test
- Eslint
- Deploy api
- Deploy api to oracle cloud again
- Ui tweaks
- Cypress and ts issue

### ⚙️ Miscellaneous Tasks

- Remove "web" from the list of workspace directories in package.json
- Add Tailwind CSS configuration files and update dependencies
- Add support for rendering Markdown content with React Markdown
- Skip API deployment for now
- Removed ./mobile
- Updated browserlist
- Update node version to 20.x in GitHub workflows
- Test deploying api to prod from CI
- Updatex deploy workflows
- Removed old code cov
- Removed old web
- Removed old ui package
- Change stage trigger to workflow dispatch

## [stg-v5.0.0-pr544.0] - 2023-04-06

### 📚 Documentation

- Update `ContributeCard` docs

### ⚙️ Miscellaneous Tasks

- Change contribution card style
- Update render test for `ContributeCard` component

## [stg-v5.0.0] - 2023-04-03

### ⚙️ Miscellaneous Tasks

- Upgrade expo to the latest version (#565)

## [stg-v4.14.2-pr540.0] - 2023-02-23

### 📚 Documentation

- Update `ProjectCard` doc

## [stg-v4.15.0-pr542.0] - 2023-01-12

### ⚙️ Miscellaneous Tasks

- Update the fetch url
- Update the title of the project card
- Update the project card with new model of the project
- Display repository urls withing the card container
- Delete `theme` property from `ProjectCard` component
- Update unit test for `ProjectCard` component

## [stg-v4.14.1] - 2022-12-17

### 💼 Other

- Add new button test

## [stg-v4.7.0] - 2022-11-16

### 💼 Other

- Add leblad go module repo

## [stg-v4.5.0-pr489.0] - 2022-11-15

### 📚 Documentation

- Add `Filter` interface doc
- Add `Option` interface doc
- Add `Project` interface doc
- Add `Route` interface doc
- Add `AppBar` component doc
- Add `BottomSheet` component doc
- Add `ContributeCard` component doc
- Add `ProjectCard` component doc
- Add `DrawerContent` component doc
- Add `ErrorBoundary` component doc
- Add `Filters` component doc
- Add `DZCodeLoading` component doc
- Add `Markdown` component doc
- Add `DrawerNav` component doc
- Add `StackNav` component doc
- Add theme doc
- Add `TryAgain` component doc
- Add jsdoc to interfaces

## [stg-v3.15.0-pr472.3] - 2022-08-07

### 🐛 Bug Fixes

- Switching to en on root was broken
- Settings not persisting in localStorage

## [stg-v3.6.0-pr448.0] - 2022-03-20

### 🚀 Features

- Added `PluginsProvider` for rtl support
- Added and `faq card` component
- Extend `jest config` setup

### 💼 Other

- Add data mockers
- Exposed box components
- Contributor dialog
- Add contributor card
- A first part of how to create a vim plugin  for a new programming language showing some  basics like project structur and  how you can create a syntax highlight using a mix of vim scirpt and regex

### ⚙️ Miscellaneous Tasks

- Reworked `faq page` with `dzcode/ui`
- Rework `folder structure`
- Added new `tests`
- Rework faq page to support new folder stucture
- Migrate team to dzcode ui

## [stg-v3.1.1] - 2021-11-14

### 🚀 Features

- Handle pages urls in multi languages
- Data code now support multi languages
- Add multilanguage support for the documentation or learn page
- Translate footer contents
- Translate static content

### 🚜 Refactor

- Refcatoring  data models
- I fixed the learn and articles links translation  issue by doing some small changes and adding the path as required prop for the sidebar , so there is no need to refactor the data code and the redux actions

## [stg-v2.13.0] - 2021-09-15

### 🐛 Bug Fixes

- Filter undifined and null committer in the github service
- Summarry or description of team controller so it can be clear for everyone , dont awaite projects it doesnt return a promise

## [stg-v2.6.0-pr336.1] - 2021-08-01

### 🐛 Bug Fixes

- Ran common:sync before api deployment

## [1.11.5] - 2021-03-31

### 💼 Other

- Copyright date auto-update

## [stg-v1.8.2] - 2021-02-04

### 🚀 Features

- A redux action to fetch document authors details from github
- Add the Authors components so we can see who is the who wrote this documentation
- Authors list to each documentation article

## [stg-v0.15.0] - 2020-10-10

### 🚀 Features

- *(new_article)* A python website no framework needed part 1:how you can create a website with python so you can understand how frameworks like flask , django ...works
- *(new_article)* A python website no framework needed part 1:how you can create a website with python so you can understand how frameworks like flask , django ...works

### 🐛 Bug Fixes

- Update title
- Remove search icon and add FAQ to navbar

### 💼 Other

- Edti title
- Typos

## [stg-v0.13.4] - 2020-09-18

### 🚀 Features

- *(new article)* A new article about using conventional commits to contribute to dzcode and other projects
- *(new article)* A new article about using conventional commits to contribute to dzcode and other projects

<!-- generated by git-cliff -->

[ZibanPirate]

Great find!

I still think the change log won't answer the design decisions questions, mainly for two reasons:

• it tells the what not the why
• it has many bullet points (some of which are not related to design decisions)

Maybe what could help is feeding this change log along with CONTRIBUTING.md into an LLM and tell it to update the design decision section based on it

[omdxp]

Great idea @ZibanPirate we may add this as part of our existing workflow when releasing a new version in our CI/CD pipeline

[yamanidev]

first decision to be recorded: https://github.com/dzcode-io/dzcode.io/blob/3341188453e1fc8ad7baa94de2a7cae1be5b7cd3/.github/CONTRIBUTING.md#design-decisions

is this how it's supposed to be used?

I think this sounds more like a commit message than a design decision documentation. The latter is explaining for example why we're using BitBucket in the first place, why use the tool X, the provider X, why take this approach in solving X problem etc...

(Apologies for the awfully late reply)

[omdxp]

@yamanidev we might think that LLM could help document these decisions based on our commits

[yamanidev]

@omdxp LLMs can't extrapolate that kind context, it'll provide generic reasons for decisions...

[omdxp]

Looks pretty good to me even though I didn't feed it the whole changelog:

Design Decisions for dzcode Project

General Design Approaches

• Standardization: Adopted consistent naming conventions across API endpoints, controllers, dictionary endpoints, and project metadata files.
• Localization: Incorporated localized labels and messages for improved user experience in multilingual contexts.
• Encapsulation: Enhanced component modularity and encapsulation, particularly in the search and TopBar components.

Version-Specific Design Decisions

Version [6.9.2]

• Workflow Optimization: Integrated pull request trigger into CI workflow.
• Resource Management: Added cleanup logic for old Docker images and containers to streamline deployment processes.
• Dependency Management: Updated MeiliSearch image and service dependencies in Docker Compose for improved functionality.

Version [6.9.1]

• Language Imports: Refactored sitemap functions to use updated language imports and types for better code readability.

Version [6.8.1]

• API Naming Consistency: Standardized endpoint names to lowercase for uniformity across the project.
• Dictionary Management: Pruned unused dictionary entries for a cleaner, more efficient codebase.

Version [6.8.0]

• Metadata Enhancement: Incorporated JSON schemas and tags into project info files to ensure proper data structure and integration.
• Data Management: Introduced methods to retrieve specific contribution metadata by ID for better backend functionality.
• UI Updates: Designed contribution URLs with XML escaping to improve sitemap accuracy.

Version [6.7.0]

• Search Integration: Enhanced the SearchService and DigestCron functionality to streamline bulk upserts, deletion operations, and search indexing.
• UI Enhancements: Added search modal and input localization in TopBar to improve usability and accessibility.

Version [6.6.0]

• Contributor Pages: Added dedicated contributor pages with detailed stats, projects, and contributions.
• API Integration: Enhanced contributor detail fetching and displayed associated projects and contributions.

Refactoring Highlights

• Code Cleanup: Removed old container management logic and redundant dictionary entries for a cleaner and more maintainable codebase.
• Search Streamlining: Improved search component behavior and encapsulated functions like keyboard event handling within dedicated components.
• Type Safety: Updated types and responses across components for improved clarity and maintainability.
• Logging and Defaults: Removed unnecessary logging and default values to ensure consistency in data processing.

Workflow and Miscellaneous Tasks

• Release Configuration: Updated release drafter templates to include comprehensive changelogs and author information.
• Index Management: Introduced setupIndexes method to ensure proper index setup and attribute updates.

Design Philosophy

• Prioritize modularity, clarity, and scalability in component development.
• Focus on usability and localization to cater to a diverse user base.
• Embrace consistent naming conventions and type safety to enhance code maintainability.

And this one with technical details:

Design Decisions with Technical Details for dzcode Project

General Design Approaches and Technologies Used

• Frontend Technologies:

  • Built using React.js for the user interface components.
  • Localization supported by react-intl for multilingual contexts.
  • State management handled by Redux for global data flow.

• Backend Technologies:

  • Server-side functionality powered by Node.js and Express.js.
  • Data retrieval optimized through MeiliSearch for search operations.
  • Utilized Cron Jobs via node-cron for periodic task automation.

• Database and Deployment:

  • Relied on PostgreSQL for structured data storage.
  • Dockerized services orchestrated using Docker Compose for scalable deployment.

• Version Control and CI/CD:

  • Integrated GitHub Actions for continuous integration and deployment workflows.
  • Automated dependency updates using Dependabot.

Version-Specific Design Decisions

Version [6.9.2]

• Workflow Optimization:
  • Modified CI workflow to include pull_request_target trigger for pre-merge testing.
• Resource Management:
  • Integrated Docker cleanup logic to prune old images and containers using Bash scripts.

Version [6.9.1]

• Code Maintenance:
  • Refactored language imports in sitemap functions for improved modularity and type checking using TypeScript.

Version [6.8.1]

• API Design:
  • Standardized endpoint names to lowercase, enhancing uniformity and readability.
• Dictionary Management:
  • Developed scripts in JavaScript to identify and remove unused dictionary keys.

Version [6.8.0]

• Metadata Handling:
  • Introduced JSON Schema for project models, ensuring validation of required properties.
  • Incorporated tags using TypeORM for relational database schema updates.
• Data Retrieval:
  • Added API endpoint for fetching contribution titles by ID using Express.js.
• UI Improvements:
  • Applied XML escaping to contribution URLs in the sitemap for compatibility with search engines.

Version [6.7.0]

• Search Integration:
  • Enhanced search functionality through bulk upsert and deletion using MeiliSearch’s API.
  • Refactored DigestCron to optimize database indexing and search operations.
• UI Enhancements:
  • Implemented a search modal integrated into the TopBar using React.js and CSS modules.

Version [6.6.0]

• Contributor Pages:
  • Developed contributor detail pages using React.js and dynamic data fetched via REST APIs.
  • Displayed associated projects and contributions using TypeScript for type-safe components.

Refactoring Highlights

• Improved Type Safety:
  • Updated entity types and responses across backend services for better clarity and maintainability using TypeScript.
• Encapsulation and Modularity:
  • Centralized keyboard event handling within the search component to improve code encapsulation.
• Search Component Updates:
  • Streamlined logic in the search component using hooks like useEffect and useState for efficient state management.
• Database Optimization:
  • Removed default values for runId in contributors and projects tables to maintain data integrity.

Workflow and Miscellaneous Tasks

• Release Configuration:
  • Enhanced release templates in GitHub Actions to include author attribution and changelog links.
• Index Management:
  • Added setupIndexes method using MeiliSearch API to configure searchable attributes and ensure index consistency.

Design Philosophy

• Adopt a modular approach to enhance scalability and maintainability across frontend and backend components.
• Emphasize type safety and consistent naming conventions to improve developer experience and code clarity.
• Ensure localization and accessibility to cater to a diverse global audience.
• Utilize modern technologies like Docker, TypeScript, and MeiliSearch for efficient deployment and data handling.