Open-Sourcing the Coveo JavaScript Search Framework

Testing. In July 2016, the Coveo Search UI, also known as the Coveo JavaScript Search Framework, became open-source. This means that, from now on, anyone will be able to go on GitHub, take the Coveo Search UI, and modify the code itself to adapt it to their own needs.

Although it is called the JavaScript Search Framework, the code is written using TypeScript. The TypeScript code is then compiled into JavaScript, which allows us to modify our code base without the fear of breaking the framework.

Why open-sourcing it?

As Coveo continues to grow, we have more and more partners that need to implement our Search UI in a website, and tweak it in different ways to fit their needs.

Before open-sourcing the code, our partners had to do some serious code-gymnastics to bypass certain sections of the Search Framework and implement their own features.

We improved the flexibility and usability of the framework, and by open-sourcing it, we are giving our advanced Coveo partners a means to take their implementation to the next level by allowing them to take the code and play with it to their heart’s content.

What’s different with the open-source version?

We did not decide to simply open-source the code; we modified it to make it more user friendly for anyone who wants to join in.

Improved documentation

Before the open-source version, the JavaScript Search Framework documentation had to be updated by hand every time there was a modification in the code. While the people at Coveo are keenly aware of the importance of good documentation, it still happened from time to time that certain features were added, removed, or modified without the documentation reflecting this change.

From now on, the documentation is generated using TypeDoc, which means that the documentation is made from comments left in the code. To access the current documentation, see the Coveo Search UI documentation.

This way, it is much easier for developers to remember to update the documentation when adding new features or modifying existing ones.

This also means that anyone playing with the open-source code is able to see the documentation right next to the component code without having to refer to an external website.

Furthermore, the TypeDoc generated documentation can be easily implemented directly in the UI components, such as the JavaScript Search Interface Editor. This way, the components can be used even by people who are not familiar with coding.

Removed internal jQuery dependencies

The old JavaScript Search Framework was dependent on the jQuery library. However, because a plethora of other packages also use this library, the potential for conflicts between the Coveo Search Framework and other packages was high.

With the new Search UI, we removed our dependency on that library, while still continuing to support it. This way, other packages or features that use the jQuery library will still work in the Coveo Search Framework, while also lowering the risk of potential conflict.

Update the way the project is built

We started using TypeScript at a really early stage in the project, nearly 3 years ago, in the JavaScript Search Framework version 0.4. Since then, the “correct” way to set up a TypeScript project has evolved tremendously.

We made extensive use of triple-slash directives to instruct the typescript compiler how to bundle the project.

While this method served us well at the beginning of the project, it has serious drawback. The main one is that the project becomes monolithic and cannot be easily customized to include or exclude certains components.

We modified our project to instead use ES6 CommonJS modules and webpack in order to have a much more flexible bundle. Technically speaking, specifying any entry point in the project results in a coherent bundle that executes correctly at run time.

This also brought some development perks, such as a webpack dev server, which makes developing with the Search UI much more enjoyable.

Can people contribute to the project?

We would love for people to contribute to our Search UI project! Simply make a Pull Request, and a feature or improvement that you coded could be added to the Coveo Search Framework.

This not only makes our developers aware of your implementations, but it also makes your other projects that much easier to start.

Coveo is very open to new ideas and features, and would love to hear from what you think should be improved. We await your pull requests in our GitHub Project, and we hope that you enjoy the new Coveo Search Framework!

Source: source.coveo.com

Using React JSX with TypeScript

In the last months, we experienced with React and we enjoyed it a lot. As you may know, all Coveo’s web applications are built using TypeScript, so with the release of TypeScript 1.6 announcing support for React JSX syntax, we were stoked!

In this article, I’ll introduce you on how to start a new project using TypeScript, JSX and React and show you some tools we use to simplify our development.

This article was updated on June 6, 2016 to use typings instead of tsd since it is now deprecated in favor of typings.

Initial setup with npm

First we’ll setup our project with npm init. For this project we need node, typescript, typings, and react. Let’s install them:

npm install typescript -g
npm install typings -g

npm install react --save

Second, let’s make sure we have TypeScript compiler 1.6 or later:

tsc --version

You should see an output similar to:

message TS6029: Version 1.6.2

TypeScript definitions with typings

We’re almost ready to start coding, but we’ll need the React definitions. We already installed typings which is a package manager to search and install TypeScript definition files directly from the community driven repositories. Most definitions are from DefinitelyTyped. DefinitelyTyped is a great project and we try to contribute as much as we can. It will allow us to download the latest definitions for React and other libraries. Like we did with npm, we need to initialize a “typings” project by running :

typings init

This will create a typings.json file (similar to a package.json but refering to our TypeScript definitions), a typings/ folder to store the definitions and a index.d.ts referencing all our downloaded definitions.

We can now install the needed definitions:

typings install dt~react --global --save
typings install dt~react-dom --global --save
typings install dt~react-addons-create-fragment --global --save
typings install dt~react-addons-css-transition-group --global --save
typings install dt~react-addons-linked-state-mixin --global --save
typings install dt~react-addons-perf --global --save
typings install dt~react-addons-pure-render-mixin --global --save
typings install dt~react-addons-test-utils --global --save
typings install dt~react-addons-transition-group --global --save
typings install dt~react-addons-update --global --save
typings install dt~react-global --global --save

This downloads the definitions to our typings folder, saves the commit hash to the typings.json and updates the typings/index.d.ts.

Our typings.json should contain something like :

{
  "dependencies": {},
  "globalDependencies": {
    "react": "registry:dt/react#0.14.0+20160602151522",
    "react-addons-create-fragment": "registry:dt/react-addons-create-fragment#0.14.0+20160316155526",
    "react-addons-css-transition-group": "registry:dt/react-addons-css-transition-group#0.14.0+20160316155526",
    "react-addons-linked-state-mixin": "registry:dt/react-addons-linked-state-mixin#0.14.0+20160316155526",
    "react-addons-perf": "registry:dt/react-addons-perf#0.14.0+20160316155526",
    "react-addons-pure-render-mixin": "registry:dt/react-addons-pure-render-mixin#0.14.0+20160316155526",
    "react-addons-test-utils": "registry:dt/react-addons-test-utils#0.14.0+20160427035638",
    "react-addons-transition-group": "registry:dt/react-addons-transition-group#0.14.0+20160417134118",
    "react-addons-update": "registry:dt/react-addons-update#0.14.0+20160316155526",
    "react-dom": "registry:dt/react-dom#0.14.0+20160412154040",
    "react-global": "registry:dt/react-global#0.14.0+20160316155526"
    //.....
  }
}

And the typings/index.d.ts should contain:

/// <reference path="globals/react-addons-create-fragment/index.d.ts" />
/// <reference path="globals/react-addons-css-transition-group/index.d.ts" />
/// <reference path="globals/react-addons-linked-state-mixin/index.d.ts" />
/// <reference path="globals/react-addons-perf/index.d.ts" />
/// <reference path="globals/react-addons-pure-render-mixin/index.d.ts" />
/// <reference path="globals/react-addons-test-utils/index.d.ts" />
/// <reference path="globals/react-addons-transition-group/index.d.ts" />
/// <reference path="globals/react-addons-update/index.d.ts" />
/// <reference path="globals/react-dom/index.d.ts" />
/// <reference path="globals/react-global/index.d.ts" />
/// <reference path="globals/react/index.d.ts" />

Let’s code

Create a file named HelloWorld.tsx. Notice the .tsx extension, this is needed for TypeScript to enable JSX syntax support.

/// <reference path="./typings/index.d.ts" />

class HelloWorld extends React.Component<any, any> {
  render() {
    return <div>Hello world!</div>
  }
}

We first reference to our TypeScript definitions that we setup in the previous step. We then import React module using the ES6 module import syntax and then, we declare our first component using react!

Compiling to JavaScript

TypeScript 1.6 has a new flag to enable JSX support, we need to enable it. Compile HelloWorld.tsx to JS by running:

tsc --jsx react --module commonjs HelloWorld.tsx


This will produce HelloWorld.js

But, you might not want to remember all those flags, let’s save our compiler configuration to a tsconfig.json. The tsconfig.json file specifies the root files and the compiler options required to compile the project. For more details refer to the official documentation.

{
  "compilerOptions": {
    "jsx": "react",
    "module": "commonjs",
    "noImplicitAny": false,
    "removeComments": true,
    "preserveConstEnums": true,
    "outDir": "dist",
    "sourceMap": true,
    "target": "ES5"
  },
  "files": [
    "./typings/index.d.ts",
    "HelloWorld.tsx"
  ]
}

We can now run tsc in our project folder to produce the same result. Notice that we include the typings/index.d.ts file, so we won’t need to reference it in all our files.

Finishing touches

Let’s explore a little deeper on how to render our HelloWorld component and pass typed props.

Let’s improve our HelloWorld component by adding firstname and lastname props and typing them with an interface. Then, let’s render it! This will allow us to be notified at compile time if a prop is missing or is the wrong type!

class HelloWorldProps {
  public firstname: string;
  public lastname: string;
}

class HelloWorld extends React.Component<HelloWorldProps, any> {
  render() {
    return <div>
      Hello {this.props.firstname} {this.props.lastname}!
    </div>
  }
}

ReactDOM.render(<HelloWorld
    firstname="John"
    lastname="Smith"/>,
  document.getElementById('app'));

Compile once again with tsc. Then let’s finish by importing everything in an index.html file:

<!DOCTYPE html>
<html>
  <head>
    <meta charset="utf-8">
    <title>React TypeScript Demo</title>
  </head>
  <body>
    

id=“app”>

 

    src="https://cdnjs.cloudflare.com/ajax/libs/react/0.14.3/react.js">
    src="https://cdnjs.cloudflare.com/ajax/libs/react/0.14.3/react-dom.js">
    src="HelloWorld.js">
  </body>
</html>

Open index.html in your browser and you should see

Hello John Smith!


That’s it! You’ve created your first TypeScript React project. Hope you enjoy developing with it as much as we do!

Note that i’ve intentionally left webpack out of this tutorial to keep it short but as your project grows to more than one file, a module loader will be necessary.

Source: source.coveo.com

Creating a new JS UI component in TypeScript

Behind the scenes, the Coveo JS UI framework is built entirely in TypeScript. Obviously, it’s intended to be customized in JavaScript, but you may want to go further and create your own component in TypeScript. Be aware that this uses the internal JS UI APIs, so it can eventually break when updating. Purpose of the post is to give a glimpse of how the components are built internally.

Huge disclaimer : I am definitely not a TypeScript expert, and I am not working in the JS UI team. It is possible that my code is not optimized. This is more to give a basic guideline 😀

Huge disclaimer #2 : This article also implies that the reader has basic notions of TypeScript and the Coveo JS UI.

If you have ever downloaded the Coveo JS UI framework, you may have noticed that there’s a folder named lib in there. This folder contains the TypeScript definitions files we will need.

A component? What are we even talking about?

The JS UI is basically made of components, which are the different parts you can simply drop in your page. It goes from the facets, sorts, result lists to more advanced stuff such as the folding or analytics. On an architectural point of view, it is important to understand that a component should have a single responsability and should (at least try to) not impact others.

So, what do we want to build?

My use case was fairly simple : I wanted to create a component I called the ToggleResultList. This component would be a simple button allowing you to have different result lists in the same page (probably with a different style) and toggle between them. The main goal is to have something I can drop in my markup like this :

<span class="CoveoToggleResultList" data-result-list="#MainResultList"data-icon="default-result-list"></span>

Where the MainResultList is the ID of the HTML element containing the result list. Something like :

class=“CoveoResultList” data-wait-animation=“fade” id=“MainResultList” data-result-container-selector=“#ResultsContainer”> …

For details on the options on the Result List, you can refer to the Developers documentation on the component.

The TypeScript frame

So, let’s start by building the most basic component we can.

/// <reference path="../lib/CoveoJsSearch.d.ts" />

module Coveo.Test {
    export class ToggleResultList extends Coveo.Ui.Component {
        static ID = 'ToggleResultList';

        constructor(public element: HTMLElement,
                    public options?: any,
                    bindings? : Coveo.Ui.ComponentBindings) {
            super(element, ToggleResultList.ID, bindings);

        }
    }
}

Coveo.Ui.CoveoJQuery.registerAutoCreateComponent(Coveo.Test.ToggleResultList);

Let’s take a small breath, look out the window, and then focus on what we just wrote. For now, it doesn’t do a single thing, but the frame is there. We start by referencing the Coveo Js Search definition file, which will allow us to compile the whole thing. Then, we create our own class that extends Coveo.Ui.Component, which is the base class for any JS UI component. We then need an ID. This will get interpreted as CoveoToggleResultList in the markup, allowing anyone to drop this element in their page.

The constructor takes 3 parameters : the actual HTML element, any options that could be set on the component (we will come back to this further) and the current bindings (such as which search interface we are in). Don’t forget to call the basic constructor!

Finally, we use the framework to register the component. This line is really important, as it will indicate the JSUI to consider your block of code as an authentic component. From now on, you could compile your TypeScript code and integrate it in your page, right after CoveoJsSearch.min.js.

Be sure to check out Will’s excellent blog post on how to create a successful build process.

Adding some functionality

We have a component, it’s kinda cool! But it would been even cooler if it actually did something… Let’s add some stuff.

/// <reference path="../lib/CoveoJsSearch.d.ts" />

module Coveo.Test {
    export interface ToggleR1esultListOptions {
        resultList: HTMLElement;
    }

    export class ToggleResultList extends Coveo.Ui.Component {
        private static coveoResultListClass = '.CoveoResultList';
        private static disabledClass = 'coveo-disabled';

        static ID = 'ToggleResultList';
        static options: ToggleResultListOptions = {
            resultList: Coveo.Ui.ComponentOptions.buildSelectorOption({ defaultFunction: () => $(ToggleResultList.coveoResultListClass).get(0) })
        };

        constructor(public element: HTMLElement,
                    public options?: any,
                    bindings? : Coveo.Ui.ComponentBindings) {
            super(element, ToggleResultList.ID, bindings);

            this.options = Coveo.Ui.ComponentOptions.initComponentOptions(element, ToggleResultList, options);

            Assert.exists(this.options.resultList);

            this.bind.onRoot(Coveo.Events.QueryEvents.querySuccess, (e: JQueryEventObject, args: Coveo.Events.QuerySuccessEventArgs) => this.handleQuerySuccess(e, args));
            $(this.element).click(() => this.handleClick());
        }

        private getClassName(): string {
            return '.' + Coveo.Ui.Component.computeCssClassNameForType(ToggleResultList.ID);
        }

        private handleQuerySuccess(e: JQueryEventObject, data: Coveo.Events.QuerySuccessEventArgs) {
            if (!$(this.options.resultList).coveo().disabled &&
                !$(this.options.resultList).is(':visible')) {
                $(this.options.resultList).show();
            }
        }

        private handleClick() {
            $(this.getClassName()).addClass(ToggleResultList.disabledClass);
            $(this.element).removeClass(ToggleResultList.disabledClass);
            $(ToggleResultList.coveoResultListClass).coveo('disable');
            $(ToggleResultList.coveoResultListClass).hide();
            $(this.options.resultList).coveo('enable');
            $(this.getBindings().root).coveo('executeQuery');
        }
    }
}

Coveo.Ui.CoveoJQuery.registerAutoCreateComponent(Coveo.Test.ToggleResultList);

As you can see, we added some options in there. Those options are interpreted as data attributes in the markup. We will then be able to associate the component to a specific result list. You may wonder why we created a static options class…

1. It’s how the JS UI components are built.
2. It makes them compatible with the interface editor, which would allow anyone to simply drag and drop your component in a new or existing search page. (that’s for the interface part)

In our options for now, we simply added a ResultList, which is a selector option (meaning it points to another HTML element). There’s a huge variety of options you can have, simply use the autocomplete feature in your favorite IDE to see them 😀 What is really important is in the constructor, we need to initialize the options, this will make the link with the component.

We then bound our component to two events : the regular jQuery click event and the JS UI’s query success event. You can refer again to the Developers documentation to learn more about these events. The click event is responsible to disabling the other result lists and then to trigger a query on the one that should be shown.
==> It is important to actually disable and not just hide other result lists, since in the case the events would stay bound on them, so it would mess with your search page state.

You may also wonder why we show the result list in the querySuccess event instead of the click one… simple : we need to make sure the query has enough time to be executed. If we were to show it right away, it would “flick” a few milli-seconds and not be enjoyable for the user.

Adding mooaaaarrrr options

So, we have a component working, isn’t this nice? If you’re really building something that other developers might use, you may want to add even more options to your component to make it really awesome.

/// <reference path="../lib/CoveoJsSearch.d.ts" />

module Coveo.Test {
    export interface ToggleResultListOptions {
        defaultResultList?: boolean;
        icon?: string;
        numberOfResults?: number;
        resultList: HTMLElement;
    }

    export class ToggleResultList extends Coveo.Ui.Component {
        private static coveoResultListClass = '.CoveoResultList';
        private static disabledClass = 'coveo-disabled';

        static ID = 'ToggleResultList';
        static options: ToggleResultListOptions = {
            defaultResultList: Coveo.Ui.ComponentOptions.buildBooleanOption({ defaultValue: false }),
            icon: Coveo.Ui.ComponentOptions.buildIconOption(),
            numberOfResults: Coveo.Ui.ComponentOptions.buildNumberOption({ defaultValue: 10 }),
            resultList: Coveo.Ui.ComponentOptions.buildSelectorOption({ defaultFunction: () => $(ToggleResultList.coveoResultListClass).get(0) })
        };

        private iconTemplate = _.template("<span class='coveo-icon <%=icon%>'></span>");

        constructor(public element: HTMLElement,
                    public options?: any,
                    bindings? : Coveo.Ui.ComponentBindings) {
            super(element, ToggleResultList.ID, bindings);

            this.options = Coveo.Ui.ComponentOptions.initComponentOptions(element, ToggleResultList, options);

            Assert.exists(this.options.resultList);

            if (!this.options.defaultResultList) {
                $(this.options.resultList).coveo('disable');
                $(this.options.resultList).hide();
                $(this.element).addClass(ToggleResultList.disabledClass);
            }

            this.bind.onRoot(Coveo.Events.QueryEvents.buildingQuery, (e: JQueryEventObject, args: Coveo.Events.BuildingQueryEventArgs) => this.handleBuildingQuery(e, args));
            this.bind.onRoot(Coveo.Events.QueryEvents.querySuccess, (e: JQueryEventObject, args: Coveo.Events.QuerySuccessEventArgs) => this.handleQuerySuccess(e, args));
            $(this.element).click(() => this.handleClick());

            this.render();
        }

        private getClassName(): string {
            return '.' + Coveo.Ui.Component.computeCssClassNameForType(ToggleResultList.ID);
        }

        private handleBuildingQuery(e: JQueryEventObject, data: Coveo.Events.BuildingQueryEventArgs) {
            if (!$(this.options.resultList).coveo().disabled) {
                data.queryBuilder.numberOfResults = this.options.numberOfResults;
            }
        }

        private handleQuerySuccess(e: JQueryEventObject, data: Coveo.Events.QuerySuccessEventArgs) {
            if (!$(this.options.resultList).coveo().disabled &&
                !$(this.options.resultList).is(':visible')) {
                $(this.options.resultList).show();
            }
        }

        private handleClick() {
            $(this.getClassName()).addClass(ToggleResultList.disabledClass);
            $(this.element).removeClass(ToggleResultList.disabledClass);
            $(ToggleResultList.coveoResultListClass).coveo('disable');
            $(ToggleResultList.coveoResultListClass).hide();
            $(this.options.resultList).coveo('enable');
            $(this.getBindings().root).coveo('executeQuery');
        }

        private render() {
            var icon = this.options.icon;
            if (icon != "") {
                $(this.element).prepend(this.iconTemplate({icon: icon}));
            }
        }
    }
}

Coveo.Ui.CoveoJQuery.registerAutoCreateComponent(Coveo.Test.ToggleResultList);

You may notice there’s now 3 more options of different types.

1. You can specify if the component is targetting the “default” result list. This means this will be the one shown by default, and the other will be hidden at the beginning (in the previous example, you would have to do it manually).
2. You can specify the number of results of the result list. Normally, you would specify it on the whole Search Interface, but you may want to display a different number according to which result list is shown. We’re hooking on the buildingQuery event to change the number of results according to the options.
3. You can specify an icon! Using an UnderscoreJS template, this is a simple commodity to render nicely an icon for your component.

Wrap it up!

Now, my real use case was to give the user the possibility to toggle between a “regular” view and a “tabular” view. My markup looks like this, where the TableResultList and MainResultList are two different elements containing the two different result lists templates :

class=“coveo-toggle-result-list-section”> class=“CoveoToggleResultList” data-result-list=“#TableResultList” data-number-of-results=“50” data-icon=“table-result-list”> class=“CoveoToggleResultList” data-result-list=“#MainResultList” data-default-result-list=“true” data-icon=“default-result-list”>

If you wonder, it is located just under the search box in a typical JS UI search page.

And the visual result looks just like this:

Thanks a lot for reading! 😀

Source: source.coveo.com