In this case, you can make use of Séntisis Webhook, which will be available in the products in which you find this functionality. By means of it, we pretend to facilitate and optimize the data export and ingestion so that no download of the mentions is needed (in .csv files).

How is the connector configured?

If you want to activate this option, you have two alternatives to do it:

Send us an email to support@sentisis.com, including the URL in which you want to receive the information - http://example.com/endpoint.
Configure the webhook from the option Integrations, placed in the dashboard, on your own.

Once the mentions are analyzed, they will be sent to the specified endpoint in real time through a HTTP POST request and they will be available for their ingestion. Furthermore, the alerts that you have configured will be also sent through the connector.

Do not forget that the mentions shall remain available in your products in sentisis.io

Which is the format of the data?

The information of the texts are sent in a JSON document with the following format, depending if they are messages that are extracted from social networks, media or alerts:

Messages of social networks (base mention)

Every mention type extends from Mention base object:

{
  id: String
  sentiment: String[POSITIVE|NEGATIVE|OBJECTIVE]
  categories: [{ id: String, name: String }]
  feed: { id: String, name: String }
  date: Date
  msgId: String
  type: String[
    Tweet|
    Retweet|
    FacebookPost|
    FacebookComment|
    InstagramPost|
    InstagramComment|
    Youtube
  ]
}

Twitter Messages

Due to the privacy policy of Twitter, the messages of this social network will include this information:

{
  user: {
    gender: String[FEMALE|MALE|UNKNOWN]
    location: {
      country: String
      region: String
      subregion: String
    }
  }
  link: String
}

You can capture the rest of the information by means of this booster

Media Messages (news, blogs & forums)

{
  title: String
  summary: String
  text: String
  source: {
    title: String
  }
  link: String
}

Facebook Messages

{
  text: String
  user: {
    id: String
    username: String
    name: String
    gender: String[FEMALE|MALE|UNKNOWN]
  }
  link: String
  isDarkPost: Boolean
  adAccount: {
    id: String
  }

Instagram Messages

{
  text: String
  user: {
    id: String
    username: String
    name: String
    followers: Number
    gender: String[FEMALE|MALE|UNKNOWN]
  }
  link: String
  isDarkPost: Boolean
  adAccount: {
    id: String
  }
}

Youtube Messages

{
  text: String
  user: {
    id: String
    username: String
    gender: String[FEMALE|MALE|UNKNOWN]
    followers: Number
  }
  link: String
}
YoutubeComment {
  text: String
  user: {
    id: String
    username: String
    gender: String[FEMALE|MALE|UNKNOWN]
  }
  link: String
}

TikTok Messages

{
  text: String
  user: {
    id: String
    username: String
    gender: String[FEMALE|MALE|UNKNOWN]
    followers: Number
  }
  link: String
}
TikTokComment {
  text: String
  user: {
    id: String
    username: String
    gender: String[FEMALE|MALE|UNKNOWN]
  }
  link: String
}

Review Messages

{
  text: String
  rating: Number
  location: {
    address: String
    locality: String
    name: String
    region: String
    link: String
  }
  user: {
    name: String
  }
}

Alerts Messages

{
  type: 'alert'
  payload: {
    id: String
    name: String
    alert_type: String[
      VOLUME |
      VOLUME_POSITIVE |
      VOLUME_NEGATIVE |
      INFLUENCE |
      INFLUENCE_POSITIVE |
      INFLUENCE_NEGATIVE |
      AUTHOR |
      AUTHOR_POSITIVE |
      AUTHOR_NEGATIVE |
      VIRALITY |
      VIRALITY_POSITIVE |
      VIRALITY_NEGATIVE
    ]
    delivery: {
      type: String
      receiver: [String]
    }
    lastnotification: String
    interval: String
    source: String[
      Facebook |
      Twitter |
      Instagram |
      Youtube |
      Media |
      Review |
      Tiktok |
      All
    ]
    authors: [String]
    message: {
      id: String
      text: String
      date: String
      userName: String
      followers: String
      source: String[
        Facebook |
        Twitter |
        Instagram |
        Youtube |
        Media |
        Review |
        Tiktok |
        All
      ]
    }
  }
}

Message examples

The data is whitewashed so that it doesn't contain real information

Twitter:

{
  "id": "5ae7b623ce6800019067b66",
  "sentiment": "OBJECTIVE",
  "categories": [
    {
      "id": "54b557ecc45f3d73a357bdf",
      "name": "Accion social"
    },
    {
      "id": "5995654ed8e3390a0f18987",
      "name": "Promocion"
    },
  ],
  "feed": {
    "id": "2aed3e8f7a411171911b346",
    "name": "Nombre marca"
  },
  "msgId": "99111360727289601",
  "type": "Tweet",
}

Media (news, blogs y forums):

{
  "id": "5ae7b623ce63800019067b66",
  "title": "Título de la noticia/blog/foro",
  "summary": "Resumen del contenido de la noticia",
  "text": "Este es el cuerpo completo de la noticia. Puede contener varios párrafos",
  "sentiment": "POSITIVE",
  "categories": [
    {
      "id": "54b557ecc545fd73a357bdf",
      "name": "Acción social"
    },
    {
      "id": "5995654ed48e390a0f18987",
      "name": "Promoción"
    },
  ],
  "feed": {
    "id": "2aed3e87a411117191b346",
    "name": "Nombre marca",
     },

  "date": "2018-04-26T12:19:11.000Z",
  "msgId": "33894752217",
  "source": {
    "title": "Nombre del medio",
  },
  "type": "Media",
  "link": "http://ct.moreover.com/?a=33894752217&p=1tg&v=1&x=EA9JefwoRHkRa223rbva"
}

Facebook:

{
  "id": "59a5842de3d02a000f8abc3",
  "sentiment": "OBJECTIVE",
  "categories": [
    {
      "id": "54b557ecc545f3d73a357bdf",
      "name": "Acción social"
    },
    {
      "id": "5995654ed48e3390a0f18987",
      "name": "Promoción"
    }
  ],
  "feed": {
    "id": "2aed3e8f7a4111171911b34",
    "name": "Nombre marca"
  },
  "date": "2017-08-29T14:02:15.000Z",
  "msgId": "155619407443876_1560144877376929",
  "type": "FacebookComment",
  "text": "Buenos días, este es el contenido del mensaje interno: nombres, cédula, mail y celular. \r\nSaludos.",
  "link": "https://facebook.com/1556194074438676_156014476929",
  "clicks": 12,
  "isDarkPost": true,
  "adAccount": {
    "id": "act_1245341172064281"
  }
}

Instagram:

{
  "id": "5942de02a000f8abc3",
  "sentiment": "OBJECTIVE",
  "categories": [
    {
      "id": "54b557ecc545f3d73a357bdf",
      "name": "Deseo de consumo"
    },
    {
      "id": "54b557ecc545f3d73a357bdd",
      "name": "Expresa risa"
    },
  ],
  "feed": {
    "id": "2aed3e8f7a4111171911b34",
    "name": "Nombre marca"
  },
  "date": "2017-06-14T14:19:37.000Z",
  "msgId": "1537044239360_628059",
  "type": "InstagramPost",
  "text": "Este es el contenido del post",
  "link": "https://www.instagram.com/p/BVUrZ-6nA/",
  "isDarkPost": false
}

Youtube:

{
  "sentiment": "OBJECTIVE",
  "categories": [
    {
      "id": "54b557ecc54f3d73a357bdd",
      "name": "Expresa risa"
    }
  ],
  "feed": {
    "id": "2aed3e8f7a4111171911b34",
    "name": "Nombre marca"
  },
  "date": "2017-08-29T16:04:29.000Z",
  "msgId": "s7r62LMc",
  "type": "YoutubeVideo",
  "text": "hola, este es el contenido del mensaje",
  "user": {
    "id": "UC5_DK5TY48EecjYfA",
    "username": "Nombreusuario",
    "gender": "MALE"
  },
  "link": "https://youtube.com/watch?v=s234u62LMc"
}

Reviews:

{
  "id": "54b557ecc54f3d73a351adb",
  "text": "Este es el contenido de la reseña de Google My Business",
  "sentiment": "POSITIVE",
  "rating": 5,
  "location": {
    "address": "Dirección de la tienda",
    "locality": "Madrid",
    "name": "Nombre de la tienda",
    "region": "Madrid",
    "link": "URL de la ubicación de la tienda en Google Maps"
  },
  "user": {
    "name": "Nombre del usuario"
  },
  "categories": [
    {
      "id": "54b557ecc545fd73a357bdf",
      "name": "Acción social"
    },
    {
      "id": "5995654ed48e390a0f18987",
      "name": "Promoción"
    }
  ],
  "feed": {
    "id": "2aed3e87a411117191b346",
    "name": "Nombre del feed"
  },
  "date": "2020-11-23T14:20:00.000Z",
  "msgId": "33894752244566",
  "type": "Review"
}

Security

Once your server is configured for receiving messages, it will receive all kinds of requests - including the ones whose source is unknown. Owing to security reasons, it is possible that you only want to receive the requests that come from Séntisis. By means of this object, we offer an optional validation system, which is based on tokens.

Webhook will calculate a hash of the body request - signature - and it will send it along with the request in a header so that you can you can programmatically verify it from this application. In order to do that, it will use functions which are irreversible and a shared secret.

Configuring the secret

Enter the Séntisis dashboard and select Webhook
Fill out the secret - any string which is between 3 and 256 characters - and save changes

Once the secret is configured, webhook will send an additional header in each request, x-sentisis-signature, including the validation token.

Validate the request

Imagine a basic server:

const express = require('express');
const bodyParser = require('body-parser');

const app = express();
const port = 4444;

app.use(bodyParser.json());

app.post('/messages', (req, res) => {
  console.log(Received ${JSON.stringify(req.body, null, 2)});
  res.status(200).end();
});

app.listen(port, () => console.log(Example app listening on port ${port}!));

The idea is to calculate the validation token in the same way that Séntisis calculates it and verify that it is equal to the one that has been sent in the x-sentisis-signature header. Séntisis uses a HMAC-SHA1 signature over the body request, so that the code would be as the following one:

const express = require('express');
const bodyParser = require('body-parser');
const crypto = require('crypto');

const app = express();
const port = 4444;

app.use(bodyParser.json());

function verifySignature(body, requestSignature) {
  const signature = crypto
    .createHmac('sha1', process.env.SENTISIS_SECRET)
    .update(JSON.stringify(body))
    .digest('hex');

  return signature === requestSignature;
}

app.post('/messages', (req, res) => {
  console.log(Received ${JSON.stringify(req.body, null, 2)});
  if (!verifySignature(req.body, req.headers['x-sentisis-signature'])) {
    res.status(500).send('Invalid signature');
  }
  res.status(200).end();
});

app.listen(port, () => console.log(Example app listening on port ${port}!));

You must adapt the code to the language that your server uses.

WARN: Remember not to share your secret or directly specify it in your code. As you can see in the example, we have used an environment variable in order to store the secret of Séntisis.

Do not hesitate to contact us if you want to use this functionality!

The use of a development/tool which enables this connection is needed.