I. INTRODUCTION
The existing broadcasting systems are starting to display dual contents from original media and the Internet as one broadcasting unit, thereby transforming into a hybrid network. By considering a second screen service, the world broadcast standards such as Hybrid broadcast broadband TV (HbbTV) and Advanced Television Systems Committee (ATSC), and Korea’s standard ground-wave networks are being altered to become the standards for the next generation of broadcasting systems [1].
While there exist several Web-related technologies for connections between devices, specialized techniques for a second screen service are quite insufficient [2]. There are still some problems related to the display of contents from multiple devices and the efficient transfer of these contents. Further, the current research on the second screen service has only been applied to testing services, and there are few, if any, successful cases of a commercial service.
Furthermore, considering the characteristics of broadcasting systems, which evolve into a hybrid broadcasting system, in this paper, we propose a second screen service model to enable the personalization of television (TV) contents by combining the existing broadcasting and Web-related techniques.
II. State of the Art
We studied the “Web and TV Interest Group” organized by W3C to find research trends related to the second screen service [3]. The Web and TV Interest Group provides a forum for Web and TV technical discussions, reviews the existing work and the relationship between Web services and TV services, and identifies requirements and potential solutions to ensure that the Web will function well with TV. Recently, research on the connection technology between various types of devices and a second screen service using Web techniques has been actively carried out in the Web and TV Interest Group’s forum [4]. Some of the major studies are the various use cases of second screen services, LinkedTV, and UPnP.
The Open Annotation Data Model(OADM) specifies an interoperable framework for creating associations between related resources, annotations, using a methodology that conforms to the Architecture of the World Wide Web through W3C Open Annotation Community Group [5].
The primary aim of the Open Annotation Data Model is to provide a standard description mechanism for sharing Annotations between systems. This interoperability may be either for sharing with others, or the migration of private Annotations between devices. The shared Annotations must be able to be integrated into existing collections and reused without loss of significant information. An annotation is considered to be a set of connected resources, typically including a body and target, and conveys that the body is related to the target. The exact nature of this relationship changes according to the intention of the annotation, but most frequently conveys that the body is somehow “about” the target. Other possible relationships include that the body is an identifier for the target, provides a representation of the target, or classifies the target in some way. RDF (Resource Description Framework) is the core technology of the OADM.
III. Design of Personalized Second Screen Service Model
Here, we propose a second screen service model to adapt Web techniques for a personalized TV contents service. The proposed model is composed of four parts. One part specifies the system configuration. Second part describes the extension design of OADM. Third part explain a method of personalized profile extraction. The last part specifies the reconstruction of additional content process.
A second screen service is a service between a public device such as a smart TV and a private device such as a smartphone. While the main contents are displayed on the public TV, the additional contents are displayed on the secondary devices. These additional contents are related to the main contents played on the TV, and the contents are changed according to the user information transferred from a second screen device to the TV. In order to play the additional contents on the second screen devices, several techniques are required to connect these devices and to deliver the contents. As shown in Figure. 1, the TV and the second screen devices are connected by using universal plug and play (UPnP) techniques, and the additional contents from the set of broadcasting contents are delivered from the TV to a second screen device by using Web socket techniques [6].
We applies OADM to the additional information representing the broadcast content for the purpose of the connection between the broadcast content and the additional content. Figure 2 shows a graphical representation of a stat of the plurality of additional content be included in a broadcast program. Annotation that are additional contents can be extended because it is the instant of class. Annotation in square box express additional content information for a specific period.
Existing OADM is not suitable to describe the resource list in the order. Although list class can represent a first and last list of the specific period of the additional contents, this class should be displayed individually in the order of the list between the first and last resource. Therefore we extend a class named OrderedList and express the list of resources in the order only by specifying the beginning and end Resources. The extension specification includes a Relationship (first, last) which can specify the first resource and last resource and Property (NumberOfItem, OrderBy) specification which specifies the number and sort of the resource files. <Table 1> is the Open Annotation Ontology Extension for new OrderedList class.
|
Class: oaExt:OrderedList URI: http://mintlab.sookmyung.ac.kr/ns/oaExt#OderedList OrderedList – A class representing a collection of resources in order. Subclass of oa:List class. This class represents the beginning and end of resource without having display all of the resources in the order. |
|
Relationship: oaExt:first URI: http://mintlab.sookmyung.ac.kr/ns/oaExt#first first – The first resource in the list that has to be expressed in ordered list. Can be represented by a resource URL. The sub property of rdf:first. |
|
Relationship: oaExt:last URI: http://mintlab.sookmyung.ac.kr/ns/oaExt#last last – The last resource in the list that has to be expressed in ordered list. Can be represented by a resource URL. The sub property of rdf:last. |
|
Property: oaExt:NumberOfItem URI: http://mintlab.sookmyung.ac.kr/ns/oaExt#NumberOfItem NumberOfItem – The total number of resources(Item) The number of resources include first and last resources. The sub property of rdf:value. Value – Integer |
|
Relationship: oaExt:OrderBy URI: http://mintlab.sookmyung.ac.kr/ns/oaExt#OrderBy OrderBy – Sort the ordered list of resource. Sub property of rdf:value. The file is limited that the file name is numbered in a sequential number or time. Value – FileName, FileTime |
The user-specific preferences analysis is needed to provide a personalized service. The user profile information is needed in order to provide user-specific additional contents[1]. It utilizes user personal information like primary TV viewing history and additional contents viewing history information in the user profile information.
The user personal information contains extracted result of the preferred program category through the user ID, age, sex, favorite category, and priority. The user personal information will utilize the values entered manually by the user in the personal device.
In primary TV viewing history information, it extracts the subject / item / Cast information of the program corresponding to the user through the top ratings of the TV program viewing pattern analysis. The more user will watch a lot of programs in a certain category, the more this category will be given a higher scores for the subject / item / Cast of the program. The ratings of a particular program (P) of the user are calculated as follows. After this, it will score the subjects / item / Cast of the program corresponding to the top of the ratings list (see Eq 1.).
Additional Contents Viewing History is same with the way of scoring calculation for Primary TV Viewing History. It is calculated separately for additional content viewing history by Purchased contents viewing points and transferred additional contents viewing points, and extracts topic / item / Cast information. In order to calculate scores in Transferred additional contents viewing history, it uses the number, the time of additional contents viewing, etc. The utilization score of Transferred additional contents will be able to calculate as follows.
(fi : Frequency of using transferred additional content, ti: Time of using transferred additional content, w1:Weight for the frequency, w2 :Weight for the time)
Purchased Content is calculated by the same way of transferred additional content. But it has a difference that weighing for the purchased content. The way of calculating score is as follows.
(fi) = Frequency of using purchased content, ti =Time of using purchased content, w1 = Weight for the frequency, w2 = Weight for the time)
The points which calculated by these three kinds of data listed in the list of each program and additional content by the topic / item / Cast. After this, add the item with the same score each other, then select K of the highest score for affinity. Like this way, it can provide affinity per user.
The reconstruction of additional content process is performed on TV. Parsing the Metadata, Annotation File and multimedia data received from the broadcast and match with user preferences. As a result, system can provide the personalized additional content according to the user. Figure 3 shows the reconstruction of additional content process.
IV. System Implementation
We show the applicability of the proposed second screen service by implementing its prototype. The prototype carries out the following three processes:
A second screen device sends a message confirming the connection on the TV screen when the TV requests a connection or when the connection is completed. In this step, the HTML connection page uses a network service discovery API to find a service in the UPnP control device [6].
UPnP consists of a control device, a controlled device, and a service. In this study, the TV is set as the controlled device because it performs the role of a server that provides the second screen device with additional contents. The second screen device is a control device playing the role of a client receiving the additional contents from the TV.
The UPnP networking process performs the following six-stage process to find the abovementioned devices and connect them: assign addresses, discover devices, device description, control, event, and presentation [3]. In the assign address stage, when the TV and the second screen devices access a common network, they search the dynamic host configuration protocol (DHCP) and receive an IP address. In the discover device stage, second screen devices use the simple service discovery protocol (SSDP) to search for a TV. When a TV is discovered, the second screen device requests for the technical description and service description of the TV device. Then, in the control stage, a second screen device analyzes the information obtained from the previous stage. The device also sends a command to the TV through the simple object access protocol (SOAP). In the event step, the TV notifies the stat of the variables and the second screen devices are operated [Figure. 4].
After the device discovery and connection step, a reconstruction process can be performed for additional contents.
The original broadcasting contents are separated into the main TV contents and the additional contents. The annotation file written in the open annotation data model is interpreted to separate the main contents and the additional contents [7]. Figure 5 shows the reconstruction process and the processes are as follow;
-
TV Signal Input → metadata, annotation file, multimedia data Parsing → Extraction of full additional contents list → Matching with User Preference → Obtaining the additional contents list for each device → Obtaining Main A/V location that corresponds to the acquired additional content → Time Reconfiguration and Content Control between Main A/V and Acquired Additional Content → Delivery of Content to Second Device /Display on TV.
An additional contents list and the corresponding menu are created through a reconfiguration process. This menu is displayed on each second screen device adaptively by using the CSS and WebSocket technology.
The menu consists of three items: List of Contents of My Interest, Current Broadcast Content, and Contents of My Interest. On the section of List of Contents of My Interest, an additional content list that matches the user’s preferences is displayed. This section’s size is flexible according to the content quantity. The Current Broadcast Content section displays a list of additional contents included in the current channel irrespective of the user’s preference. This section can be automatically updated whenever a TV scene changes. The Contents of My Interest section plays the contents selected by the user from the List of Contents of My Interest menu or the Current Broadcast Content menu.
After the second screen device and the TV are connected, the additional contents can be sent to the viewer’s second screen device according to the viewer’s request by using WebSocket[8]. WebSocket enables a real-time transmission of the additional contents whenever the TV scene is switched. Next, we implement a server that can send binary data with the Node.js module and the contents to a client’s webpage through WebSocket. In this section, we describe the server and client configuration and the implementation method for the server/client side for real-time contents delivery.
A server is built into the TV and a webpage plays a broadcast on the TV. The TV server executes a series of functions and stores the additional contents data. When the TV sends the segmented file number corresponding to the current channel to the TV server, the TV server verifies the existence of the additional contents. If the file has additional contents, the server sends a list of these additional contents to the second screen application according to the user’s preference. The list is sorted and displayed in an appropriate section. Then, the selected contents are streamed through WebSocket from the TV server. Figure 6 shows the configuration of the server and the client.
Next, we create a server that performs binary data communication with the BinaryServer() function in the server-side javascript. It designates a variable that defines the current situation and executes certain functions according to the situation. When a connection is established, the server receives the metadata file from the client. At this instance, the file is handled with the pipe() function.
In the client-side javascript file, the IP address of the server that will connect to the BinaryClient() function becomes a factor for connection. Further, the protocol for WebSocket communication is written with “ws,” and not “http.” When the statement of a client becomes “open,” the stream and streaming file is set by the createStream() function.
V. Implementation of Sample Prototype
In this section, we suggest a scenario for the realization of a personalized service and confirm the results of the sample prototype.
We present a service scenario that can be applied to an actual Korean TV program. This scenario introduces a service that encourages a viewer to stay on the current channel. In this scenario, there are three viewers watching a music program; two viewers have common interests, and the third viewer has different interests. While the viewers are watching the music program, various additional services such as preview video, paid contents, and program information are provided on the individual devices according to the viewer’s interest. Figure 7 shows the flow of the scenario.
There are three viewers. Viewer A is a person who likes the group “EXO” and uses a tablet as the second device. Viewers B and C do not have a specific favorite singer, but they usually like watching a music program on the TV. However, the second screen devices of these two viewers are different; viewer B uses a laptop, and viewer C uses a smartphone.
On viewer A’s second screen device, EXO’s rehearsal preview video is delivered, while EXO is not on the stage. Viewer A purchases and watches the full version of EXO’s rehearsal video waiting for the EXO’s on-stage time. Upon EXO’s turn, a notification is sent to viewer A’s screen. When EXO’s video is being played on viewer A’s screen, the program and cast information contents are delivered on viewer B’s and viewer C’s screens. Although viewers B and C are provided the same contents, the layout of these contents varies according to the device size.
We set the precondition for the sample prototype of the proposed second screen service scenario. A laptop is used instead of a TV in the hybrid network environment. This laptop stores additional contents on its server. Further, we assume that the TV receives the user’s profile from a second screen service application.
The prototype execution is as follows: The connection buttons on the services pages of the second screen devices enable these devices to access UPnP. When the saved segment files start playing, the laptop sends a message to the second screen devices. This message shows that the laptop has each of the second screen devices’ contents. If a user selects the desired contents from the menu, a request to assign the additional contents for real-time streaming is sent to the server. If the server accepts this request, the related additional contents are streamed.
VI. CONCLUSION
In conclusion, we confirmed the basic Web-related technology for the realization of a second screen service, such as the connection technology between different devices and the real-time contents transmission technology. Then, we designed a service model and a service scenario for a personalized second screen service and confirmed the possibility of this service through the prototype implementation. In the process, we make it easy to express a large amount of content by extending the OADM class. This study hopes for a more advanced and efficient technology to extract Web contents from a set of broadcasting contents and deliver them to the user-desired device. Moreover, we expect the advanced second screen service to be commercialized in the future.
