• IP addresses are NOT logged in this forum so there's no point asking. Please note that this forum is full of homophobes, racists, lunatics, schizophrenics & absolute nut jobs with a smattering of geniuses, Chinese chauvinists, Moderate Muslims and last but not least a couple of "know-it-alls" constantly sprouting their dubious wisdom. If you believe that content generated by unsavory characters might cause you offense PLEASE LEAVE NOW! Sammyboy Admin and Staff are not responsible for your hurt feelings should you choose to read any of the content here.

    The OTHER forum is HERE so please stop asking.

Serious Not DSTA but GovTech Data Scientists actual solved the Circle Line Mystery!

Faker

Alfrescian
Loyal
Not bargin Hen's DSTA, why he claimed fake credit for his fake DSTA 'ENGINEERS'?

http://www.straitstimes.com/singapore/transport/national-data-scientists-explanation-of-circle-line-investigation-a-fascinating


GovTech data scientists' explanation of Circle Line investigation 'a fascinating account': PM Lee




Published
Dec 1, 2016, 9:56 pm SGT
Updated
Dec 2, 2016, 4:38 pm

SINGAPORE - The raw data was voluminous: lists of which trains in the Circle Line were stalling, where and at what times.
But plotting the information as a chart revealed a pattern. Trains seemed to be breaking down one after another, almost as though malfunctioning was contagious.
Data scientists from the Government Technology Agency of Singapore (GovTech), which was set up on Oct 1, have released a summary of the process that led them to identify the "rogue train" behind extensive train disruptions on the Circle Line earlier this year.



"When a train got hit by interference, another train behind moving in the same direction got hit soon after," data scientist Daniel Sim wrote in a post published on GovTech's blog on Thursday (Dec 1) afternoon.
"What we'd established was that there seemed to be a pattern over time and location: Incidents were happening one after another, in the opposite direction of the previous incident. It seemed almost like there was a 'trail of destruction'."



Mr Sim, who holds a computer science degree from the University of Cambridge and another from ETH Zurich, wrote that the team of data scientists hit upon a hypothesis: "Could the cause of the interference be a train - in the opposite track?"
15003345_1162624907164100_8422796050899076353_o.jpg

15003345_1162624907164100_8422796050899076353_o.jpg


Related Story
Meet the defence engineers who cracked the mysterious Circle Line case




This brainwave, when applied to the data set, pointed to the existence of one rogue train - which the data scientists identified by traipsing down to Kim Chuan Depot to review video footage of train movements.
Prime Minister Lee Hsien Loong and Defence Minister Ng Eng Hen have publicly commended the scientists who got to the bottom of the problem.
Mr Lee, in a Facebook post on Thursday, singled out the GovTech data scientists' explanation of their research as "a fascinating account, demonstrating close teamwork, sharp analysis, and a never-say-die attitude".
"Proud of the team's good work, and a big thank you to all the officers who worked so hard to crack the puzzle!" said Mr Lee, who has a bachelor's degree in mathematics and a diploma in computer science from the University of Cambridge.
He has described himself as deriving pleasure from computer programming and previously made headlines around the world for sharing the code to a Sudoku-solving algorithm that he wrote.
The disruptions were investigated by a multi-agency team that included engineers from the Land Transport Authority and train operator SMRT; a DSO National Laboratories team led by engineer Chian Teck Keong; Defence Science and Technology Agency engineers Mui Whye Kee, Cheng Heng Ngom and Benedict Koh; and GovTech scientists Lee Shangqian, Daniel Sim and Clarence Ng.
 

Faker

Alfrescian
Loyal
15003345_1162624907164100_8422796050899076353_o.jpg


SINGAPORE - Engineers Mui Whye Kee, Cheng Heng Ngom and Benedict Koh were the Defence Science and Technology Agency (DSTA) experts who helped solve the mysterious glitch which affected the MRT Circle Line in recent weeks.

Mui Whye Kee, Cheng Heng Ngom and Benedict Koh are directors and manager, not ENGINEERS as reported.

Mui Whye Kee
Director
PROGRAMME CENTRES
COMMAND, CONTROL, COMMUNICATION, COMPUTER AND INTELLIGENCE DEVELOPMENT
https://www.gov.sg/sgdi/ministries/m...epartments/c4i

Mr Cheng Heng Ngom
Deputy Director (C4I Development)
Defence Science and Technology Agency
https://www.mindef.gov.sg/imindef/mi.../winners2.html

Benedict Koh
Senior Systems Manager at DSTA
https://sg.linkedin.com/in/benedict-koh-61023614
 

Faker

Alfrescian
Loyal
16004_p1_s_1.jpg



https://blog.data.gov.sg/how-we-caught-the-circle-line-rogue-train-with-data-79405c86ab6a#.tf11tsxxd

[h=1]How the Circle Line rogue train was caught with data[/h]Text: Daniel Sim | Analysis: Lee Shangqian, Daniel Sim & Clarence Ng
The MRT Circle Line was hit by a spate of mysterious disruptions in recent months, causing much confusion and distress to thousands of commuters.
Like most of my colleagues, I take a train on the Circle Line to my office at one-north every morning. So on November 5, when my team was given the chance to investigate the cause, I volunteered without hesitation.



From prior investigations by train operator SMRT and the Land Transport Authority (LTA), we already knew that the incidents were caused by some form of signal interference, which led to loss of signals in some trains. The signal loss would trigger the emergency brake safety feature in those trains and cause them to stop randomly along the tracks.
But the incidents — which first happened in August — seemed to occur at random, making it difficult for the investigation team to pinpoint the exact cause.
We were given a dataset compiled by SMRT that contained the following information:

  • Date and time of each incident
  • Location of incident
  • ID of train involved
  • Direction of train
We started by cleaning the data. We worked in a Jupyter Notebook, a popular tool for writing and documenting Python code.
As usual, the first step was to import some useful Python libraries.


Snippet 1We then extracted the useful parts from the raw data.
Snippet 2We combined the date and time columns into one standardised column to make it easier to visualise the data:
Snippet 3This gave us:
1*Z6u8o3FiIH7kUEuJxY-4xQ.png


Screenshot 1: Output from initial processing


[h=3]No clear answers from initial visualisations[/h]We could not find any obvious answers in our initial exploratory analysis, as seen in the following charts:
1. The incidents were spread throughout a day, and the number of incidents across the day mirrored peak and off-peak travel times.


1*KX-DNuL7rcE-to9chy0m1g.png


Figure 1: Number of occurrences mirror peak and off-peak travel times.2. The incidents happened at various locations on the Circle Line, with slightly more occurrences on the west side.
1*oLktbZBwhnI31V86FL6ddw.png


Figure 2: The cause of the interference did not seem to be location-based.3. The signal interferences did not affect just one or two trains, but many of the trains on the Circle Line. “PV” is short for “Passenger Vehicle”.
data-frequency-by-train-id-data.png


Figure 3: 60 different trains were hit by signal interference.


[h=4]The Marey Chart: Visualising time, location and direction[/h]Our next step was to incorporate multiple dimensions into the exploratory analysis.
We were inspired by the Marey Chart, which was featured in Edward Tufte’s vaunted 1983 classic The Visual Display of Quantitative Information. More recently, it was used by Mike Barry and Brian Card for their extensive visualisation project on the Boston subway system:


data-marey-chart-data.png


Screenshot 2: Taken from [url]http://mbtaviz.github.io/[/URL]In this chart, the vertical axis represents time — chronologically from top to bottom — while the horizontal axis represents stations along a train line. The diagonal lines represent train movement.
We started by drawing the axes in our version of the Marey Chart:
1*uspR3Ymc1d1HA-hTVfa3tg.png


Figure 4: An empty Marey Chart, Circle Line versionUnder normal circumstances, a train that runs between HarbourFront and Dhoby Ghaut would move in a line similar to this, with each one-way trip taking just over an hour:
data-ccl-marey-chart-data.png


Figure 5: Stylised representation of train movement on Circle LineOur intention was to plot the incidents — which are points instead of lines — on this chart.



[h=4]Preparing the data for visualisation[/h]First, we converted the station names from their three-letter codes to a number:

  • Marina Bay to before Promenade: 0 to 1.5
  • Dhoby Ghaut to HarbourFront: 2 to 29
If the incident occurred between two stations, it would be denoted as 0.5 + the lower of the two station numbers. For example, If an incident happened between HarbourFront (number 29) and Telok Blangah (number 28), the location would be “28.5”. This made it easy for us to plot the points along the horizontal axis.


Snippet 4And then we computed the numeric location IDs…
Snippet 5And added that to the dataset:
Snippet 6Then we had:
1*sdWUDnDhDHM4oFxgH0lG6w.png


Screenshot 3: Output table after location IDs are addedWith the data processed, we were able to create a scatterplot of all the emergency braking incidents. Each dot here represents an incident. Once again, we were unable to spot any clear pattern of incidents.
1*AXOCrqn4S6FXA-paleLmWg.png


Figure 6: Signal interference incidents represented as a scatterplotNext, we added train direction to the chart by representing each incident as a triangle pointing to the left or right, instead of dots:
data-ccl-disruptions-data.png


Figure 7: Direction is represented by arrows and colour.It looked fairly random, but when we zoomed into the chart, a pattern seemed to surface:


Figure 8: Incidents between 6am and 10amIf you read the chart carefully, you would notice that the breakdowns seem to happen in sequence. When a train got hit by interference, another train behind moving in the same direction got hit soon after.



[h=3]How can signal interference move through a tunnel?[/h]At this point, it still wasn’t clear that a single train was the culprit.
What we’d established was that there seemed to be a pattern over time and location: Incidents were happening one after another, in the opposite direction of the previous incident. It seemed almost like there was a “trail of destruction”. Could it be something that was not in our dataset that caused the incidents?
Indeed, imaginary lines connecting the incidents looked suspiciously similar to those in a Marey Chart (Screenshot 2). Could the cause of the interference be a train — in the opposite track?


1*TgghmP2ljL1tOeDKu8WP6Q.png


Figure 9: Could it be a train moving in the opposite direction?We decided to test this “rogue train” hypothesis.
We knew that the travel time between stations along the Circle Line ranges between two and four minutes. This means we could group all emergency braking incidents together if they occur up to four minutes apart.
Snippet 7We found all incident pairs that satisfied this condition:
Snippet 8We then grouped all related pairs of incidents into larger sets using a disjoint-set data structure. This allowed us to group incidents that could be linked to the same “rogue train”.
Snippet 9Then we applied our algorithm to the data:
Snippet 10These were some of the clusters that we identified:
[{0, 1},
{2, 4},
{5, 6, 7},
{8, 9},
{18, 19, 20},
{21, 22, 24, 26, 27},
{28, 29, 30, 31, 32, 33, 34},
{42, 44, 45},
{47, 48},
{51, 52, 53, 56}]Next, we calculated the percentage of the incidents that could be explained by our clustering algorithm.
Snippet 11The result was:
(189, 259, 0.7297297297297297)What it means: Of the 259 emergency braking incidents in our dataset, 189 cases — or 73% of them — could be explained by the “rogue train” hypothesis. We felt we were on the right track.
We coloured the incident chart based on the clustering results. Triangles with the same colour are in the same cluster.
1*nnhR-6L3YfvoZ79Pwp30Ag.png


Figure 10: Incidents clustered by our algorithm


[h=3]How many rogue trains are there?[/h]As we showed in Figure 5, each end-to-end trip on the Circle Line takes about 1 hour. We drew best-fit lines through the incidents plots and the lines closely matched that of Figure 5. This strongly implied that there was only one “rogue train”.


data-ccl-rogue-train-marey-chart-data.png


Figure 11: Time of clustered incidents strongly implies that the interference could be linked a single trainWe also observed that the unidentified “rogue train” itself did not seem to encounter any signalling issues, as it did not appear on our scatter plots.
Convinced that we had a good case, we decided to investigate further.



[h=3]Catching the rogue train[/h]After sundown, we went to Kim Chuan Depot to identify the “rogue train”. We could not inspect the detailed train logs that day because SMRT needed more time to extract the data. So we decided to identify the train the old school way — by reviewing video records of trains arriving at and leaving each station at the times of the incidents.
At 3am, the team had found the prime suspect: PV46, a train that has been in service since 2015.



[h=3]Testing the hypothesis[/h]On November 6 (Sunday), LTA and SMRT tested if PV46 was the source of the problem by running the train during off-peak hours. We were right — PV46 indeed caused a loss of communications between nearby trains and activated the emergency brakes on those trains. No such incident happened before PV46 was put into service on that day.
On November 7 (Monday), my team processed the historical location data of PV46 and concluded that more than 95% of all incidents from August to November could be explained by our hypothesis. The remaining incidents were likely due to signal loss that happen occasionally under normal conditions.
The pattern was especially clear on certain days, like September 1. You can easily see that interference incidents happened during or around the time belts when PV46 was in service.


data-pv46-data.png


LTA and SMRT eventually published a joint press release on November 11 to share the findings with the public.



[h=3]Final thoughts[/h]When we first started, my colleagues and I were hoping to find patterns that may be of interest to the cross-agency investigation team, which included many officers at LTA, SMRT and DSTA. The tidy incident logs provided by SMRT and LTA were instrumental in getting us off to a good start, as minimal cleaning up was required before we could import and analyse the data. We were also gratified by the effective follow-up investigations by LTA and DSTA that confirmed the hardware problems on PV46.
From the data science perspective, we were lucky that incidents happened so close to one another. That allowed us to identify both the problem and the culprit in such a short time. If the incidents were more isolated, the zigzag pattern would have been less apparent, and it would have taken us more time — and data — to solve the mystery.
Of course, we were most pleased that all of us can now take the Circle Line to work with confidence again.
 

Attachments

  • 1*nnhR-6L3YfvoZ79Pwp30Ag.png
    1*nnhR-6L3YfvoZ79Pwp30Ag.png
    17.6 KB · Views: 837
Last edited:

eatshitndie

Alfrescian (Inf)
Asset
tested for over a week and sextified with findings and results. no more work, shake leg, balik kampung liao. sinkie teams are very dedicated and talented. sinkies must thank the pap. circle line now runs like colon totally cleansed with probiotics.

image.jpg
 

Seee3

Alfrescian (Inf)
Asset
First was DSTA and now this with such so much detail. Haha, is it a rogue train or rogue passengers? Still trying to understand how the rogue train can do it?
 

mojito

Alfrescian
Loyal
tested for over a week and sextified with findings and results. no more work, shake leg, balik kampung liao. sinkie teams are very dedicated and talented. sinkies must thank the pap. circle line now runs like colon totally cleansed with probiotics.

View attachment 29110

So sinkie travel on circle line is to what travels in colons huh.
 

Faker

Alfrescian
Loyal
First was DSTA and now this with such so much detail. Haha, is it a rogue train or rogue passengers? Still trying to understand how the rogue train can do it?

Can you believe one director, one deputy director and a senior manager actually solved the mystery in a short time? It would practically take them months, given their roles is management in nature and not technical. DSTA has a lot of engineers, their bosses did the HARD? work and what are the ground engineers doing?
 
Last edited:

Seee3

Alfrescian (Inf)
Asset
Can you believe one director, one deputy director and a senior manager actually solved the mystery in a short time? It would practically take them months, given their roles is management in nature and not technical. DSTA has a lot of engineers, their bosses did the HARD? work and what are the ground engineers doing?
I am skeptical of the all the reports given. They claimed "some interference, resulting in train losing signal and so causing emergency braking". Even shutting down mobile network for testing. Does signaling system work this way or misinformation to divert the attention. Anyway, credit has to be given that they realised that problem is coming from the train in the opp direction.
 

Faker

Alfrescian
Loyal
I am skeptical of the all the reports given. They claimed "some interference, resulting in train losing signal and so causing emergency braking". Even shutting down mobile network for testing. Does signaling system work this way or misinformation to divert the attention. Anyway, credit has to be given that they realised that problem is coming from the train in the opp direction.

Hen was double quick to claim credit for his DSTA and fed reporters with inaccurate information through his facebook page. Why is he so snake slow to claim back our terrex vehicles?
 

virus

Alfrescian
Loyal
Hen was double quick to claim credit for his DSTA and fed reporters with inaccurate information through his facebook page. Why is he so snake slow to claim back our terrex vehicles?

u cant handle the truth, imagine if he come clean to say whorejinx wear the pant and pinkie can be manipulated by billy bush also except he dont have the nuclear code to blast at china.
 

Faker

Alfrescian
Loyal
u cant handle the truth, imagine if he come clean to say whorejinx wear the pant and pinkie can be manipulated by billy bush also except he dont have the nuclear code to blast at china.

Hen is kicked out of the PAP's CEC 2017-2019, and a defence minister is a very important cabinet minister. It can't be due to terrex-gate as it still ongoing, probably is the way he was too over quick to claim sole credit for DSTA which is under him and over praising the fake defence "engineers" who are actually very high management staff and pushing other contributing governmental and private agencies to the sideline. Govtech decided not to keep mum and poured out the truth in great details to the mainstream and social media, their charts are very chiam meh and I don't really understand them!



The one wearing blue t-shirt is the only DSTA engineer in the picture and the rest are management team of at least senior manager level trying hard to understand the finding presentation by the engineer. The picture looked staged.
 
Last edited:

Seee3

Alfrescian (Inf)
Asset
so if it is a train with driver got such problem?

Yes and no. Yes in the sense that the system will still activate emergency brake. No as the system can be isolated and train is controlled purely by the driver, so cannot go too fast.
 
Top