Design and Implementation of Data Storage System
Using USB Flash Drive
in A Microcontroller Based Data Logger
Oka Mahendra, Djohar Syamsi, Ade Ramdan
Indonesian Institute of SciencesResearch Center for Informatics Bandung, Indonesia
[oka, djohar, ade]@informatika.lipi.go.id
Marcella Astrid
Information and Communication Technology Faculty Multimedia Nusantara University
Tangerang, Indonesia [email protected]
Abstract—Although Universal Serial Bus (USB) Flash Drives are commonly used as a portable storage medium, the use of them for a remote microcontroller based data logger are still rare. This paper discusses the design and implementation of data storage system using a USB Flash Drive on a microcontroller-based data logger. An expandable data logger was made microcontroller-based on microcontrollers with a master-slave system. A master board handles the entire processes that are including data acquisition, storage, and communication. The data storage system board acts as a slave using ALFAT OEM board and a USB Flash Drive. Its electronic schematic is described and its algorithm is explained. Some tests carried out to prove the design and implementation works properly. The test result showed 3.6% error in 2 minutes saving interval and 2.4% error in 5 minutes saving interval.
Keywords—USB Flash Drive; data storage; data logger; microcontroller.
I. INTRODUCTION
Data logger is needed to collect information from sensors and then save it to data storage. Data logger is used in many fields, such as astronomy [1], industrial [2], and medical [3]. There are many ways to make data logger, such as using one microcontroller that is connected to sensors and the save it by itself [4], or using more than one microcontroller that each microcontroller has its task called slave microcontroller. Master manages the communication between slaves. Master and slaves can be connected via RS-485 bus [3], [5] or wireless medium [2]. This research uses a master-slave system with RS-485 communication. ATmega32 is used for a master and other slave microcontrollers are used for data acquisition and data storage system.
There are many data storage available in markets. The most common one is USB Flash Drive because of its user-friendliness, size, and compatibility. Usually, USB Flash Drives are used for generic portable storage devices, not for a data logger purpose. USB Flash Drive compatibility is better compared to SD card because USB port 2.0 is compatible with USB device 3.0 or vice versa. SD port is not compatible with SDHC or SDXC. Because of USB Flash Drive user-friendliness, not all microcontrollers can be USB host. There is
a process called enumeration. Enumeration is a process where USB host choose which type of device that is connected and how the data is transferred based on the device type. There are three modes data transfer in USB: interrupt mode, bulk mode, and isochronous mode [6].
To create data storage to USB, microcontroller that support USB host or USB OTG (On-The-Go) is needed. Examples of their use in the data logger application are using Atmel’s AT90USB1287 [7], AT90USB647 [8], and Microchip’s PIC24FJ256GB110 [9]. These data logger implementations are strictly using specific microcontrollers. For microcontroller without USB host/OTG support, we need an interfacing board such as Parallax Inc.’s Vinculum Embedded USB Host Controller IC [10] and GHI Electronics. ALFAT OEM Board/ALFAT USB board [11] are also can be used as an interfacing board.
This proposed system in this paper uses ALFAT OEM Board to allow using any microcontroller that does not have USB host or USB OTG for accessing USB drive. This implementation gives more flexibility to choose microcontrollers. The proposed system will be used in the weather station built by authors. The current weather station system uses a Multi Media Card (MMC) or Secure Digital (SD) Card.
This paper describes software and hardware design of master and slave boards for data storage system using USB Flash Drive. Reliability of this system such as a probability of data loss and incorrect data for 2 and 5 minutes interval is tested.
The remainder of this paper is organized as follows: section 2 describes the methodology of the system, including hardware and software design; section 3 explains the results of some tests performed and its analysis; section 4 concludes the paper.
II. METHODOLOGY
This section describes the methodology of the system. The explanations consist of the rules of slave ID and the communication, the design of all systems, the design of 2015 International Conference on Automation, Cognitive Science, Optics, Micro Electro-Mechanical System, and
Information Technology (ICACOMIT), Bandung, Indonesia, October 29–30, 2015
expandable data logger master, the design of slave data storage to USB Flash Drive, and the design of sensor slave board with ADC.
In this research, we built an expandable data logger consists of master and three types of slave, as shown in Fig. 1. Master and slave are connected via RS-485. Slave data storage to USB Flash Drive is connected to ALFAT OEM Board via SPI. Master is connected to RTC via I2C.
Fig. 1. The proposed expandable data logger system; master and slaves are built using AVR ATMega microcontroller
Some features of expandable data logger are: (1) saving voltage value from sensor slave board’s ADC pin to USB Flash Drive, (2) sending voltage value from sensor slave board’s ADC pin to server via modem, (3) a real time clock (RTC), (4) connection to up to 10 slaves, (5) changing, adding, deleting slave ID, and (6) interval of features (1) and (2) that can be changed by user. Interval minimum is 2 minutes and maximum 24 hours (has not tested yet). The interval that is saved and inputted is in minute unit.
Slave ID is needed so the master can communicate with the correct slave. Slave ID is saved both in master and slave. Master can save up to 10 slave IDs in its EEPROM. If the program is modified, the number of slave IDs can be increased up to the size of EEPROM. Slave ID has maximum four characters that divided into two parts: one character for defining the type of slave, and up to three number characters that unique from other same-type slaves. If slave ID is started with character ‘U’, then that slave is for saving to Flash Drive (storage slave board). If slave ID is started with character ‘M’, then that slave is connected to modem (modem slave board). If slave ID is started with character ‘S’, then that slave is connected to sensor (sensor slave board).
Total number of ID that each type has is 1,110 because number of unique ID can be one character to three characters. There are ten slave IDs can be made from one character, 100 slave IDs can be made from two characters, and 1,000 slave IDs can be made from three characters. The one that limits this number to ten slave IDs is the size of master’s EEPROM.
Before the master gives commands to the slave, the master must initialize the slave with commands *<slave ID>*. Slaves aren’t doing anything except waiting for master to initialize. As shown in Fig.2, slaves compare slave ID that master sent with
its slave ID that is saved in EEPROM. If slave ID that master sent is same with slave ID in the slave, then that slave sends @OK@. After the master receives @OK@, then the master can give commands to that slave.
To change slave ID from master, firstly, the master initializes the slave that we want to change its slave ID. Secondly, after initialization successes, the master gives command ^<new slave ID>^. As shown in Fig. 2, if the slave receives character ‘^’ after initialization, the slave catches the new slave ID. The new slave ID must has same slave type than the old one. If not, the slave sends @FAILED@. The slave sends @OK@ after saving the new slave ID to EEPROM. If master receives @OK@, master will save the new slave ID to EEPROM.
Users can find out slave ID using serial RS-232 communication and HyperTerminal by typing “??”. As shown in Fig. 2, when slave receives “??”, slave will send its slave ID via serial.
The block diagram of master can be seen in Fig. 3. the block diagram shows that master has the following components: (1) one microcontroller ATmega32A, (2) one keypad 4x4, (3) one LCD 20x4, ad (4) DS1307 as RTC. LCD and keypad 4x4 are used as user interface so users can access the menu.
Fig. 3. Schematic of master board
Normally, as shown in Fig. 4, when the master doesn’t display menu and communicate with slave modem or slave data storage to Flash Drive, master counts the interval and display current time taken from RTC. When the time comes, master calls sensorToModem() and sensorToUSB() functions. In sensorToModem(), the master takes data from each sensor slave board and send it to each slave modem. In sensorToUSB(), master takes data from each sensor slave board to each slave data storage to USB. This paper does not discuss sensorToModem() function. Master counts interval by converting hour, minute, and second to integer and then modulus it by interval. The formula is (second + 60*minute + 3600*hour)%(interval*60). If the modulus result is zero or less than the last result, then do the functions. The user can access the menu after press ‘#’ from a keypad. After finish, the program counts the interval again.
The master saves data to USB Flash Drive with format “full directory\<slave ID sensor>\<year with format YY>\<month with format MM>\<date with format DD>.CSV”. We use .CSV format so it can be opened by Microsoft Excel. The date is taken from RTC.
In sensorToUSB() function as shown in Fig. 5, the master searches for slave ID for sensor and slave ID for data storage to USB Flash Drive. If either doesn’t exist in master’s EEPROM, the process will be terminated. If they exist, master initializes sensor slave board. Master can give 2 types of character to command sensor slave board in this function. They are character ‘]’ to ask 1 byte of data and character ‘!’ to ask the 1 byte previous data because the received data is invalid. Those
characters are used to hold sensor slave board so it will not send data when master has not ready to receive. Each sensor data contains seven characters. The first character is a ‘+’ or ‘– ‘sign. The fourth character is a decimal separator. Other characters are number character. Other than that, the data is invalid. Master will save eight sensor data and then initializes slave data storage to USB Flash Drive. Master will send a command “SAVEUSB#0#<number of char of full directory>#<full directory>#<number of char of data>#<data> “ part by part or byte by byte when slave data storage send character ‘[‘. Number 0 is USB port number that is used. In the current system, the user can’t change the port. It is needed because the slave doesn’t have enough memory to save all of command to a variable. Each command part received; the slave processes it first to ALFAT. Even the data isn’t saved by slave. After the slave receives one byte, the slave sends it to ALFAT. Also, if the slave receives invalid data (not invalid command), slave will send ‘x’ character. When the master received ‘x’ character, it will send the previous data.
Fig. 4. Flowchart of master’s main program
microcontroller wants to get data, Rx interrupt is activated, and microcontroller counts the loop. If microcontroller gets the data, it gets out from the loop and returns the character it got. When the loop is done without any data received, then timeout occurred.
Fig. 5. Flowchart of master’s sensorToUSB()
For ADC input in sensor slave board, we are using seven 4K7 resistors as divider circuit as shown in Fig 6. 5V will be divided into 8 different voltage values. Sensor slave will convert 10 bit ADC value into 0 – 5 volt in float. In manual calculation, each pin value can be seen in Table 1. Those values will be compared with the data saved in Flash Drive. Error rate will be compared when the data was saved in 2 minutes interval and when the data was saved in 5 minutes interval.
TABLE I. MANUAL CALCULATION VOLTAGE DIVIDER
PIN Voltage PA.0 (4700*7)/(4700*7)*5 = 5V
PA.1 (4700*6)/(4700*7)*5 = 4.29V PA.2 (4700*5)/(4700*7)*5 = 3.57V PA.3 (4700*4)/(4700*7)*5 = 2.86V PA.4 (4700*3)/(4700*7)*5 = 2.14V PA.5 (4700*2)/(4700*7)*5 = 1.43V PA.6 (4700*0)/(4700*7)*5 = 0V PA.7 (4700*1)/(4700*7)*5 = 0.71V
As shown in Fig. 6 and Fig. 7, there are LCDs used in the slave for data storage board and the slave for sensor board.
Fig. 6. Schematic of slave board for data acquisition
Fig. 7. Schematic of slave board for data storage system
The reset circuit components are a NPN transistor, a 330 Ω resistor connected to its collector, a 4K7 Ω resistor connected to its base. Its emitter is connected to ground. Also sometimes, the slave needs to reset ALFAT when ALFAT can not receive commands.
In Fig. 7, pull-up resistors are needed to switch ALFAT into SPI mode. To make ALFAT turn into SPI mode, ALFAT’s SPI_SSEL and SPI_MOSI pin must in a high state when reset. Before this, pull-up is connected to 5V analog. But, after system integration, the ALFAT can’t turn into SPI mode. Then 5V analog is changed to digital so the pull up resistors are connected to port C.1 that that always high.
III. RESULT AND DISCUSSION
Some tests were performed to make sure the function of the system is running properly. Normally, data storage in Flash Drive started on 13th seconds after the time comes. Time needed for sensorToModem() process is 13 seconds. When the slave modem is not activated, data is saved in 30th seconds.
Data storage tests were performed in 2 minutes and 5 minutes interval as shown in table 2. The data stored in USB Flash Drive were compared to manual calculation of divider circuit ± 0.01%. If they are outside that ranged, it is considered as error. Number of error and probability of error are shown in Table 2. Probability of error is higher about 1% in 2 minutes interval than in 5 minutes interval. But, the errors don’t more ±0.02 than manual calculation.
The system also tested in 2 minutes interval without activating other slave. Normally, without the slave modem, the data is saved in 30th seconds. The probability of data loss is 3/423 = 0.71%. After analyzed further, the data lost after 126 and 127 data. After 126 or 127 data, Flash Drive initialization failed and ALFAT needs to be reset. Because the initialization failed, save is also failed in this slave data storage. But, the slave can be modified so after reset, it doesn’t failed but still receives command from master.
TABLE II. ERROR DATA RATE IN 2 MINUTES AND 5 MINUTES INTERVAL
This paper proposed a data logger system using master-slave microcontroller with a USB drive as a storage device.
The communication process between master and slave must be controlled so when master wants to communicate with a certain slave, the other slaves don’t interfere. The system saves voltage value from sensor slave, sends them to web server via modem, and save them to USB Flash Drive. This paper also discussed mechanisms that keep the system from freezing and reduce memory usage because of too many variables.
The system is more reliable in a longer interval as seen in lower error probability in 5 minutes interval than 2 minutes interval. The test result showed 3.6% error in 2 minutes saving interval and 2.4% error in 5 minutes saving interval. The slave algorithm for data storage system can be modified so the data loss after resetting ALFAT because of fail to initialize Flash Drive is minimized or even eliminated.
Users also still have to manually input slave ID to the master when new device is attached and manually delete the slave ID when device is detached. The system would be better if the master can auto-detect devices that attached or detached to the system.
REFERENCES
[1] Vishnu S.N.K.N., Jeevan B., Rathan K.G.A, and Hegde R., “Weather Data Logger Based On Pic Microcontrollers for Astronomical Site Survey,” International Journal of Knowledge Engineering, Vol. 3, Issue 2, pp.-212-215.
[2] S. Venkatlakshmi, S. Vivekanandhan, S. Revathi, E. M. S. Arul, and C. Paramasivam, “Industrial Process Management Using LabVIEW,” IOSR Journal of Electronics and Communication Engineering (IOSR-JECE), Vol. 5, Issue 6, March-April, pp. 46-56, 2013.
[3] C. Kavitha, A. V. Ramana, and S. S. Raj, “Embedded Management System For Out Patient Department,” International Journal of Embedded Systems and Applications (IJESA), Vol. 2, No. 3, Sept., pp. 47-56, 2012.
[4] A. Kumar, I. P. Singh, and S. K. Sud, “Development of Multi-Channel Data Logger for Indoor Environment,” Engineering, Vol. 2, No. 3, June, pp. 690-697, 2010.
[5] R. Gupta and M. Awad, “PIC Controller Based Process Control System using GSM,” International Journal of Advanced Research in Computer Engineering & Technology, Vol. 1, Issue 5, July, pp. 79-82, 2012. [6] M. Brain, “How USB Ports Work”, howstuffworks.com, para. 1, .
[Online]. Available: http://computer.howstuffworks.com/usb3.htm . [Accessed Sept. 3, 2014].
[7] A. Prayoga, “Rancang Bangun USB On-The-Go Enabler dengan Mikrokontroler AT90USB1287”, S1 Thesis, Universitas Indonesia, Depok, Indonesia, 2010.
[8] Atmel, “8-bit Atmel Microcontroller with 64/128Kbytes of ISP Flash and USB Controller”, AT90USB646, AT90USB647, AT90USB1286, AT90USB1287 datasheet, 2012.
[9] S. Jaiswal, S. Dakh, and S. M. Sonawane, “USB Driver Embedded Controller Using Embedded C,” International Journal of Research in Advent Technology, vol. 2, no. 5, May, pp.423-427, 2014.
[10] P. Shah, N. Shah, S. Shah, and V. Mohite, “USB Portable Console Using Vinculum II Tool Chain,” International Journal of Research in Advent Technology, Vol. 1, No. 3, Oct., pp. 89-94, 2013.
