Public Lab Research note


GSoC 2021 Proposal: Translation System Refinements

by govindgoel | March 25, 2021 19:45 25 Mar 19:45 | #26033 | #26033

About me

Name: Govind Goel

Personal Email: govindgoel82@gmail.com

College Email: govindgoel@am.students.amrita.edu

Github: https://github.com/govindgoel

Affiliation: Amrita School Of Engineering, Amritapuri Campus

Location: Amritapuri, Kerala, India

Timezone: Indian Standard Time (UTC +05:30)

Project description

In plots2, an internationalization (i18n) system exists but it lacks a consistent workflow for importing new translations and it has issues related to HTML parsing and UI shortcomings.

Abstract:

The internationalization feature which has been implemented in publiclab.org(plots2) is great and it can allow users to view the content of the site in their native language. It is implemented using rails-I18n gem with translations arranged in YAML files and loaded into views using the custom translation helper function.

Problem

What problem(s) does your project solve?

Currently, the internationalization system exists based on the rails-i18n, and a custom translation function is implemented but it leads to problems related to CSS alignment, missing translations, and improper HTML parsing so it's difficult to see proper localization in publiclab.org which trouble the users across the globe due to lack of multi-language support. All of these problems will be solved by this project.

Note: All the mockups/prototypes and code samples are just for reference. They can be revised/updated according to the requirements of the project, moderator preferences, or suggestions given by mentors and the community.

Project Goals and its implementation

1. Refining custom translation helper

We need to refine this translation helper. The existing helper looks something like thisFig no.1 Existing custom translation helper

1.1 Error 404 page

Overview:
Currently, the globe icon directly redirects to the Transifex page which in turn results in an error 404 page for a user who is not in the translation team of Public Lab on Transifex.


Fig no.2 404 page

So instead of directly linking the translation helper, we can use the below approach

Direct the translation helper to the translation dashboard or wiki and there the user can know everything regarding how the whole internationalization process works. Then a request can be made to join the translation project on Transifex. So if a user receives a 'translation-helper' tag we can map the helper icon link to redirect to the Transifex project for that particular user.

1.2 Writing translation helper function in JS

The custom translation helper function can be implemented in JS, which will allow testing the usability of both functions. Similar test cases can also be implemented for both functions.

2. Resolving HTML parsing issues

2.1 Issue with placeholder

Overview:

There seems to be an issue with the placeholder we are using in the input text box, which also leads to there being no translation in websites. This is because one of the attributes skipped over by browsers is the placeholder causing the placeholder to remain as the originally authored language. This issue leads to populating HTML tags. The translation helper only provides the locale strings no HTML tags.

Linked Issue: https://github.com/publiclab/plots2/issues/7798

Resource: Link

Implementation:

To fix this behavior we can use a label instead of a placeholder to solve the issue as a label will also do the same work. The advantage of using labels is that labels don't get erased when you enter data in the input box, helping users remember the use of the input box.

The below example is of the file app/views/dashboard/_wiki.html.erb where the existing placeholder method has been replaced with label and the preview is also there along with it.

2.2 Using custom CSS over bootstrap to fix parsing issues.

Overview:

Using bootstrap for tooltip along with translation helper lead to parsing issues. As few bootstrap functions are only able to parse plain text so this leads to the issue and breaks the button view at places where the tooltip is used.

Implementation:

So instead of using bootstrap class for tooltip. We can use custom CSS for the tooltip to be shown on the hover of a button. We can also modify the tooltip as per our needs and it will fix the parsing issues as well. Below is an implementation for the same.

The below changes need to be implemented in app/views/dashboard/_wiki.html.erb here the existing tooltip function of bootstrap is removed and CSS is used for the functionality and possible fix for the issue.

This is the custom CSS that can be defined in a button.css file so that we can reuse it wherever it is needed to fix the issue.

So to include the CSS in a page we can import the file with its name in the stylesheet tag.

Example:

<%= stylesheet_link_tag "button" %>

3. Improving CSS of missing translation prompt

The CSS of the translation prompt needs to be fixed in the below mentioned components for a better user/translator experience. This is an important issue to address as these components are reused throughout the website.

As of now, the translation prompt is rendered in a different a tag, moving it to the a tag of headings in the header will fix the issue. So this issue arises due to the rendering of the translation prompt.

3.1 Potential Fixes Needed :

  • Header

In the header the translation prompt icon is not aligned along with the title it extends the header and it is rendered below a particular heading.

  • Lists all the wiki

On the page where all the wikis are listed the helper icon has a similar issue as it doesn't get aligned properly it shifts to a new line.

  • Buttons

In the search dialog box and buttons for publishing here also there is an issue with the translation helper as it is not rendered properly. It goes out of the defined area of buttons and dialog boxes.

  • Dropdowns

The issue in most of the dropdowns is that the helper icon is shifted below a heading, which doesn't look good and troubles the user/translator.

3.2 Limiting rendering of helper icon per page

As of now, the helper icon is rendered for every translation which doesn't exist so it can be limited with the use of CSS pseudo-selectors to 4-5 per page.

Fig no.18 Existing

Fig no.19 Changed

3.3 Show box or dotted lines over text that is not translated and helper on hover

Overview:

Currently, most of the users don't know about the translation system and also that they can contribute to the process of translating. So there is a need for notification to users so they can know about it. Another approach can be to display a notification on the dashboard after signing up or log in.

Implementation:

Using CSS this functionality can be easily implemented and it will surely enhance user experience as the helper function will be only shown on hover and also the text which needs to be translated will be recognized very easily.

Fig no.20 Dotted line over missing translation text

3.4 Notification to users so they know about translation helpers and can contribute.

As of now, it might happen the user might not know about the translation so adding a notification will surely attract the user's interest and attraction towards the translation of strings.

The notification can be displayed irrespective of the language chosen as there might be users who know different languages.

4. Translation dashboard for admin and to recruit translators

Overview:

As of now, there is no page to recruit translators that can help in accumulating translations for the website in different languages that are supported as of now.

Implementation:

(This is a preview of a prototype for the page) Below is the new page that will be implemented as part of the project -

Header:

The same header to be followed.

Translation Guide:

This section will list all the details such as - regarding the translation projects and status of work done and with all the prior info to be looked at.

Interest Form:

This section will contain a form for those who are interested in joining as a translator, they need to provide basic info in the form so that the details can be reviewed and verified, after which the process will be carried on.

We can collect user/translator emails, their native language, and any other languages they know.

Footer:

The same footer to be followed.

User WorkFlow

Fig no.24 User WorkFlow

Admin WorkFlow

Fig no.25 Admin WorkFlow

We can also use the existing infrastructure to achieve the goal as currently, the existing infrastructure consists of:

  • Translation wiki which provides all the information regarding the translation process, workflow, and demos.
  • Translation tag which a user can follow to stay updated with all the discussions and resources regarding the translation in Public Lab.

Changes to be done in existing infrastructure

  • Workflow And Guide [Section-2]:
    • As of now, the guide exists on the wiki page we can work on revamping it by adding translation workflow in Transifex and Github, internationalization process, and updating the existing translation guide.
  • Stats For Admin[Section-3]:
    • Transifex already has an interactive set up to show statistics about users and translation progress. If we need to implement the same on the Public Lab site as well we can set up GET API calls to get data using Transifex API and visualize it.

A dashboard visualizing the completion of the translation of strings, stats, or on what all pages translation is implemented/left.

Code Snippet

Recruiting translators[Section-4 ]:

We can implement this by two approaches:

  1. We can develop a basic form for recruiting translators, which collects user email, the language they are fluent with, and resume/cv if needed. Then the admin/moderator can review the application and then invite the translator to the Transifex project to begin translating.
  2. The wiki page already has a guide that shows how one can start translating by joining Transifex. So we can keep the same process as the existing one i.e to ask interested users to join the translation team, get the 'translation-helper' tag, then send a request to join the Transifex translation team on the Transifex site itself, thus the request can be further reviewed and approved by the admin and the translator can begin translating.

Discussion/FAQ Section:

We can make use of the existing comment functionality that exists in the translation wiki also

5. Document and provide workflow guidance for code tasks related to translation import

Overview:

  • There is a need for documentation as the first thing a new contributor/developer does is look at how to get started with setting up the dev environment in their local machine. Hence the steps involved should be documented so that new contributors have no issues while setting it up. The same goes here documenting how the internationalization system works will surely help contributors, translators so one can easily know what all steps are involved and how to modify or contribute regarding translation import. It will help other folks who might want to implement internationalization on their platform so they can also get a brief idea about it.

Implementation:

Fig no.27 Github and Transifex Workflow

  • Document in the Readme itself how the internalization works in the platform. How one can get started with it and it will cover the whole process right from adding a new language or making edits in existing translations.

  • For Example, setting up a guide in the internationalization section in Github on the steps to add, work or modify the .yml files containing translations:

    • Create a new ##.yml file while replacing ## with 2 or 3 letter language code.

  • Then these are set to the views files as shown below.

  • Steps for modifying any existing translation values or strings.
  • Steps for modifying or removing translation keys.
  • Process of importing translations from Transifex
    • There is a need to set up hassle free import of translations from Transifex to avoid missing any translation and make the process hassle-free.
    • The Transifex guide covers most of the things required to set up imports to Github. There are several ways:

6. Improve workflow for FTOs based on the translation

6.1 Easily Self Test Translation

Overview:

As testing translations from the PR can be time consuming and will lead to a slow development process of translation so implementing self test of translations with the help of APIs, Github Actions will make the job and process much more smooth and hassle free.

Implementation:

  • Run a job in Github Action
  • Get diff of the PR

Example: https://patch-diff.githubusercontent.com/raw/octocat/Hello-World/pull/900.diff

  • Check with en.yml translated into required languages.
  • Use an API like Transifex, Google Translate to check if the translation is correct or not.

6.2 Prepare pull request for review

Overview:

  • There is no template and steps one can look at to open a PR for adding translation in the .yml files, there is a need to document everything and make the process easier for the contributor.

Implementation:

  • A new template for translation issues, the addition of a translation tag to the issue. And guidelines in the PR template if it is for translation. It will help in classifying translation issues and PR. Translators can comment on the PR itself or through a bot and also GitHub suggests requesting a review from a specific team to get it reviewed by the translator community and can be merged thereafter.

Fig no. 30 Sample Translation PR Template

  • As we already have a first-timers bot, a stale bot integrated with the Github projects so similarly, we can also integrate a Github Bot to comment on the translation-specific issues, PR's, and requesting comments from the translator team.
  • Adding a translator issues section in community-toolbox, which will help a user to go through the specific issues and help in boosting up the translation process.

Fig no. 31 translation helper issues

Implementation in the community-toolbox:

We can push the issues with the translation label in the translation helper-related issues section, similar to as done for the first-timers-only labeled issues.

WorkFlow to prepare pull request for review:

Fig no.32 WorkFlow for PR

7. Add tests for newly created pages as part of internationalization

7.1 Tests to check all links are functional and direct to the correct pages.

Overview:

The functional test will be introduced for the new pages and will be implemented in the directory test/functional/ as introducing these will make sure that these pages don't break with any new changes in the near future.

Implementation:

The functional test to check new pages are rendered on the route defined can be defined as below.

7.2 Tests to check components placeholder/labels because in some languages the length of the word extends after translation, so it is essential to see how the button names display to make sure the translated word will fit.

Implementation:

This is an important test that needs to be introduced as while translating a particular string the translated one might be large and which might break any UI component such as text in buttons, search bar across the website.

In case the locale is Hindi we can implement the test in this way. So that for every locale it is assured that translated word for in the UI.

7.3 Check that the date-time format and currency symbol displayed correctly.

7.4 Introducing a test page where all scenarios can be tested as a whole.

  • Rendering of custom translation helpers in different scenarios.
  • Labels are mapped according to the translated string and don't break the UI.
  • The date and time format are correct.
  • The tooltip doesn't interrupt the helper function.

This is needed as there are many instances where translation happens so testing major things on one page will provide a brief overview of everything is working correctly.

Tests will be introduced in a new .rb file in test directory under system[test/system/translation_helper_test.rb]

8. Places where the translation is not added.

  1. https://publiclab.org/post: As this page is very crucial and it helps the community members to create posts, edit their writeup so there is a need to introduce translation on this page.
  2. Footer: Footer of the website needs to be translated as the header.
  3. Tags needed to be translated which are commonly used

Timeline/milestones

Creation of first-timer-only issues

first-timer-only issues are issues meant for first-time contributors in open source. New folks in open source, those who want to work along with the community require first-timer-only issues to easily get started. Thus, I plan to create FTOs during both phases. Also, I'll always look for any issue that can be converted to a first-timers-only issue wherever possible.Needs

I will require guidance from my mentors, suggestions, insights, and feedback from the members of Public Lab will be great and will help me to build, complete my project and contribute to the community.

Contributions

I started my OSS contribution by working on the first-timers-only issue in plots2, and since then the Public Lab community has been there to back me. I have been an active member of the Public Lab for the past year. I have also served as a mentor for the last year GSoC and also I have done a good number of contributions in the Public lab not just in plots2 or just by contributing through code and pull requests but being an active member by reviewing PRs, helping other fellow contributors to set up and fix issues along with them, participating in discussions, welcoming new contributors and much more.

Links :

Comments

Issues [plots2] Issues [community-toolbox]

PR [plots2] PR [community-toolbox]

Reviewed PR's


Experience

I am a 2nd-year student at Amrita School Of Engineering, Amritapuri Campus, and have been doing web development right from the first year itself. I have done many projects in JavaScript, React, and Ruby. I have also served as a GSoC mentor past year in PublicLab.

Projects I have worked on in the past year are mentioned below.

amFOSS Website - I have worked on migrating the codebase from GatsbyJS to NextJS.

amFOSS CMS - CMS is a management system to manage club members such as their daily updates, leave track records, achievements. I am also one of the top contributors to cms.

WorkFit - Built under a hackathon, the main goal of the project is to keep you fit while working.

CollegeBuddy - Connect with your fellows in college. Discuss, plan and organize anything on the platform. This project was also selected in Kharagpur Winter of Code, where I helped new contributors to taste the working of open source.

ChatApp - A basic chat app based on socket.io, react, express


Teamwork

I have participated in various team competitions at different levels. I have participated in hackathons where working along with the team is the key. I have also participated and helped in organizing various college tech events along with the members of the amFOSS club which is a free and open source club of our college. Regarding PublicLab I have gained a lot of experience and guidance from every member of the PublicLab especially @jywarren, @bansal_sidharth2996 @cess. Working in Public Lab has been a great journey for me till now and I hope the same for the future as well.


Passion

I started working with Public Lab as I really loved the concept of Public lab and the work they are doing for the environment and also for the folks who are new to open source. Having a platform where people can share their ideas and research work related to the environment to other projects such as editor, image-sequencer. This shows the diversified nature and care for the environment. This is all that motivates me to work along with the community.


Audience

This project will help all the users as it will provide them with the ability to translate the website content to their native language. People from remote places who care for the environment and are eager to collaborate, discuss so they will be able to make use of publiclab.org in their day-to-day life. Getting content in their native language will help them to navigate and onboard easily on the platform and make the best use of it. It will also help in increasing user daily activity.

Commitment

Yes, I fully understand that this is a serious commitment and I can devote 30-35 hours weekly for the completion of this project. Also, I don't have any major plans during the period and I will keep the mentors up to date with the university timeline while providing a work progress report every week to keep them updated with the work.


8 Comments

Hi there, @warren, @cess, @bansal_sidharth2996, and of course to all the community members, I have published the draft proposal for GSoC 2021 for the Translation System Refinements in plots2. I am still working on it and improving out the solutions and implementations as I am gaining insights. Everyone's feedback and review are most welcome. Thanks :) Warm Regards, Govind Goel

Reply to this comment...


Thank you for sharing your ideas @govindgoel, I have some feedback or comments:

  1. For the first point under Resolving HTML parsing issues please expound more, is what is currently happening related to not detecting an HTML attribute? Are you able to share the code that is not doing this?

  2. In the images shared from the Potential Fixes Needed: Header it would be better to describe what each screenshot is and what is happening in each screenshot, I had difficulties understanding which one represents the CSS issues or pointing them out.

  3. Are there screenshots or images you'll be uploading under this point Page to recruit translators It was difficult to understand what this statement meant without the preview (This is a preview of a prototype for the page) Below are the changes that will be done as part of the project -

Also, consider looking at the Public Lab style guide as you create the preview https://publiclab.org/notes/warren/05-07-2019/introducing-a-draft-style-guide-for-public-lab

  1. I did not see you address this point we need an additional "guide" on the process of importing translations from Transifex, in your project proposal from the issue description https://github.com/publiclab/plots2/#internationalization

Some things you could consider, whether accessibility features are needed, will you create first-timer issues from your project to welcome new users?

Thanks again for sharing your proposal, all the best!

Is this a question? Click here to post it to the Questions page.

@ruthnwaiganjo Thanks for the detailed insights and the areas I need to improve. I am working on them and will surely update everything asap. Again thanks a lot. 😃 Warm Regards, Govind Goel


Reply to this comment...


Hi! thank you so much for your proposal!

On 1. Refine the custom translation helper a few thoughts I had were:

  1. we do this in both JS and Ruby, i believe - that is, the i18n gem has a translation helper in each, and i believe we've overridden the Ruby one (using translate() now instead of t()), but we should think on how to do this in JavaScript as well! If it's simple and well written enough, we can just port over the code so both work normally.
  2. we might want to run similar tests on both the JS and Ruby versions
  3. for the display issues where the globe is showing up oddly, i wonder if we might want to consider a different display. What if we instead added a ....... dotted underline to words that needed translation, and when you hover over with the mouse, a globe icon appears? This might be easier to format. However, if there is already a hover-over popover or tooltip, we'd have to ensure that the new one doesn't interrupt it or overlap it.

Thanks! I'm going to copy these comments into the main discussion issue too so everyone can reflect on these ideas!

Is this a question? Click here to post it to the Questions page.

Thanks a lot, @warren for the detailed review and points that I should consider in the proposal. And regarding the display issues, we can surely implement the functionality as it will make the UI much more cleaner, and also it would be easier to identify which all strings are leftover. Considering hover-over popover or tooltip beforehand and also with the help of tests covering the major scenarios, it can be made sure that the globe icon doesn't conflict with any other UI component.


Reply to this comment...


As for the recruitment process, i changed the title of this issue to think about how to smooth the process for first-time translators. Maybe this could be part of the plan too! https://github.com/publiclab/plots2/issues/9394

Reply to this comment...


Hi, just wanted to note that I left a comment about the recruiting and community coordinating portion of the project, which I like that you've begun to sketch out here -- just some ideas and thoughts you might like to read over as i noted on this other proposal! https://publiclab.org/notes/YogeshSharma01/04-04-2021/gsoc-proposal#c28481

Thanks!

Reply to this comment...


And be sure to check out my thoughts on a test suite for different translation helpers in my comment here! https://publiclab.org/notes/ajitmujumdar25999/04-05-2021/gsoc-proposal-2021-translation-system-refinement#c28483 -- thanks!

Reply to this comment...


Login to comment.