Computer vision with Microsoft Cognitive Services

March 20, 2017 | 1 Minute Read

A few months ago I discovered Microsofts Cognitive Services which is a bunch of ML empowered APIs that makes it REAL easy to add e.g. computer vision to your product. Of course, someone already made a neat little wrapper around the APIs as a R package, Roxford (Apparently the project was previously called Project Oxford).

I took the Roxford package for spin and fired up a quick shiny app for testing the computer vision API. Give it a try here! or copy the code below and make your own, just create and insert your own key and use a Microsoft API of your liking.

library(shiny)
library(Roxford)
library(shinythemes)

# Build the ui part -------------------------------------------------------

ui <- shinyUI(
  fluidPage(theme=shinytheme("superhero"),
            sidebarLayout(
              sidebarPanel(
                titlePanel("Insert the image URL"), h5("Copy and paste e.g. this url and hit send"),
                textInput("url", label = h5("http://static.wixstatic.com/media/8dc02a_c68673e648754a7593b66cbfb5e1f6f5~mv2.jpg")), submitButton("Send"),
                hr(),
                fluidRow(verbatimTextOutput("text"))),
              mainPanel(
                fluidRow(column(3, htmlOutput("image"))))
            )
  )
)

# Build the server part ---------------------------------------------------

server <- shinyServer(function(input, output) {

  visionkey <- 'xxxxxxxxxxxxxxxxxxxxxxxx'

  output$text <- renderPrint({
    t <- getDescriptionResponseURL(input$url, visionkey)

    t$captions.text <- as.character(t$captions.text)

    t$captions.text[1]
  })

  output$image <- renderText({c('')})

})

# Run the app -------------------------------------------------------
shinyApp(ui, server)