Objectives: To study the accuracy and reliability of two point of care devices (Hemocue 201 [HC201]) and Hemoglobin Colour Scale (HCS) with reference to automated laboratory analyser Sysmex XP 100 (LabSXP) and determine factors influencing their performances. To discuss certain design issues while ascertaining these parameters for its judicious use in both clinical context and disease burden studies. Methods: Reliability and accuracy statistics were calculated for four population subgroups that were selected randomly using cluster sampling in a rural community of eastern India. Appropriate measures were taken to reduce biases in the study. Bland Altmann Plot was used to determine Bias and ROC curve analysis was used to suggest new cut-offs for HCS method. Results: True prevalence varied across subgroups ranging from 12.56% in adolescent boys to 40.71% in adult women. Sensitivity for HC201 was lowest among boys (80.39%) and highest among adult females (92.82%), while specificity was highest among adult males (86.94%) and lowest among adult females (75.00%). The variation across subgroups was due to differences in distribution of underlying Hb values. HC201 has potential for use in clinical practice as well as disease burden estimation study. HCS was not suitable for use in clinical setting as the bias (-4.1 g/dl) was unacceptable. However, ROC analysis suggested certain cut-offs for different age groups and can be employed in poor resource settings for disease burden estimation study. Conclusions: HC201 is better than HCS as per the study. However, accuracy parameters are likely to vary depending on the distribution of underlying Hb distribution. Thus, same machines can demonstrate different accuracy parameters in different settings. There is need for designing studies that could help estimate these parameters each time on a subsample which would go a long way in efficiently using technology be it for guiding clinical decisions or public health actions.